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ABSTRACT  (CONTINUED) 

The  SMARTTS  characteristics  address  the  instructor,  the  trainees,  and  the 
training  system  management.  They  provide  the  instructor  with  added 
capabilities  to  develop  training  exercises,  conduct  the  training  process, 
monitor  the  performance  of  the  trainees,  and  provide  specific  instructional 
guidance  to  the  trainees.  They  assist  the  trainees  by  providing  a  wide 
range  of  relevant  information  via  visual  displays  prior  to,  during,  and 
following  participation  in  a  scenario  on  the  trainer.  A  basic  level  of 
management  support  is  provided  in  the  form  of  long-term  data  collection,  and 
the  information  for  development  and  structuring  of  training  programs. 

Major  capabilities  developed  as  part  of  the  SMARTTS  training  subsystem 
include:  (a)  automated  assistance  for  exercise  development;  (b) 
automatically  generated  and  recorded  performance  indicators;  (c)  visual 
feedback  displays  for  presentation  of  performance  indicators  and  tactical 
variables,  in  the  attack  center  and  in  the  classroom;  (d)  alternative 
tactics  capability  for  rapid  generation  of  alternative  sets  of  actions;  (e) 
automatic  instructor  cues  based  on  scenario  events;  (f)  student  entered  time 
flag,  automatically  recorded,  to  indicate  points  of  concern  during  replay; 
(g)  automatic  interactive  target  providing  computer  control  of  a  target 
platform,  responding  to  ownship  actions;  (h)  an  exercise  library  containing 
a  large  number  of  exercises,  each  of  which  can  be  automatically  loaded  and 
started  in  the  system;  (i)  simultaneous  and  independent  operation  of 
classroom  and  attack  center  subsystems;  (j)  fast-time  and  real-time 
classroom  simulation  subsystem  for  support  of  pre-scenario  and  post-scenario 
briefing  sessions  (e.g.,  feedback);  (k)  exercise  playback  presenting  a  wide 
variety  of  tactical  variables  and  performance  indicators  as  occurring  during 
the  scenario;  (1)  an  instructors  console  located  near  the  fire  control  party 
in  the  attack  center,  providing  a  wide  range  of  monitoring  and  control 
capabilities  for  the  instructor;  (m)  an  instructors  handbook  addressing  the 
development  and  conduct  of  an  effective  training  process;  and  (n)  various 
documentation  supporting  the  development  of  SMARTTS  and  its  operation.  This 
report  provides  a  summary  overview  of  SMARTTS,  including  its  development  and 
resultant  characteristics. 

A  research  and  development  plan  has  been  assembled  to  recommend  the  next 
stages  of  effort  appropriate  for  the  continued  development  of  training 
assistance  technology. 
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EXECUTIVE  SUMMARY 


The  Submarine  Advanced  Reactive  Tactical  Training  System  (SMARTTS)  has 
been  developed  as  the  'training  subsystem*  of  the  21A41A  submarine  combat 
system  trainer.  The  SMARTTS  design  is  that  of  a  generic  training  subsystem, 
which  combined  with  the  traditional  simulation  subsystem  and  associated 
tactical  equipment,  form  the  training  device.  SMARTTS  embodies  a  variety  of 
training-related  features  that  are  essential  to  an  effective  training 
process.  The  generic  capabilities  embodied  in  SMARTTS  are  required  on  every 
training  device,  to  a  greater  or  lesser  extent,  and  should  be  tailored  to 
the  particular  aspects  of  each  training  situation. 

The  SMARTTS  concept  is  the  embodiment  of  training  assistance  technology 
(TAT)  in  the  simulator-based  training  device,  as  an  integral  part.  Whereas 
the  typical  sophisticated  military  training  device  is  actually  only  a 
complex  simulator,  doing  a  relatively  good  job  of  copying  the  operational 
environment,  the  training  device  with  SMARTTS  actually  has  capabilities 
designed  to  address  the  training  process.  The  training  device  is  more  than 
just  a  simulator;  it  should  provide  the  trainees  with  more  than  just 
experience;  it  should  provide  automated  capabilities  to  assist  the 
instructor  and  interface  with  the  trainees,  thus  enhancing  the  training 
process.  The  current  state  of  training  technology  reveals  a  wide  range 
concepts  (e.g.,  feedback)  critically  affecting  the  training  process.  The 
complexity  of  military  systems  and  operations  precludes  instructors  from 
manually  taking  advantage  of  many  aspects  of  current  training  technology. 
The  computer-based  processing  and  interface  capabilities  currently  available 
in  most  simulator-based  training  devices,  however,  enable  the  automation  of 
many  of  these  training  technology  concepts.  Hence,  the  training  assistance 
technology  embodied  in  SMARTTS  is  "good  generic"  training  technology  adapted 
to  and  integrated  with  the  inherent  processing  capabilities  of  the 
sophisticated  training  device.  These  training  process-related  capabilities 
make  up  the  training  subsystem  cf  the  training  device  to  augment  the 
traditional  simulation  subsystem. 

The  essence  of  SMARTTS  is  its  capability  to  collect  and  generate  a 
variety  of  information  pertinent  to  the  tactical  training  problem,  and  to 
display  various  combinations  of  that  information  on  the  instructor's 
console,  classroom  large  screen  display  and  the  attack  center  overhead 
monitors  at  appropriate  times  auring  the  training  process.  Substantial 
support  capabilities  are  resident  in  the  system  to  manipulate  the 
information  in  the  various  ways  required  during  the  training  process.  It 
should  be  noted  that  the  preprototype  SMARTTS  does  not  incorporate  all  of 
the  possible  characteristics  recommended  in  earlier  analytic  studies. 
Rather,  the  preprototype  unit  includes  those  characteristics  essential  to 
the  SMARTTS  concept  of  training  assistance  technology  which  were  feasible 
within  budget  and  time  constraints.  It  does  provide  the  basic  structure  for 
implementing  additional  characteri st ics  or  application  to  other  training 
situations. 

The  21A41A  SCST,  including  tne  SMARTTS  addition,  addresses  the  training 
of  combat  teams  of  the  SSN  688  class  submarine,  using  the  MK 117  fire  control 
system.  The  SMARTTS  preprototype  installed  at  Norfolk,  Virginia  is  capable 
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of  either  standalone  operation  or  operation  integrated  with  the  traditional 
21 A41  SCST  in  support  of  attack  center  training  and  classroom  training. 
SMARTTS  hardware  consists  of  an  instructor's  remote  console  (IRC)  located  in 
the  attack  center  to  monitor  and  control  various  aspects  of  the  problem,  and 
two  overhead  CRT  displays  in  the  attack  center.  A  similar  instructors 
console  is  located  in  the  adjacent  classroom.  It  controls  a  large  screen 
display  in  the  classroom.  8oth  the  classroom  and  attack  center  subsystems 
can  be  operated  prior  to,  during,  and  following  an  attack  center  exercise. 
The  intention  of  this  design  is  to  support  a  variety  of  tactics  training 
needs  across  multiple  courses  and  levels  of  trainees. 

Performance  indicators  have  been  developed  to  evaluate  the  trainee's 
responses  and  provide  corrective  feedback.  Each  performance  indicator 
addresses  a  specific  aspect  of  the  tactical  situation;  examples  of  Pis  are 
probability  of  counterdetection,  and  target  motion  analysis  initial  range 
estimate.  The  basis  of  the  SMARTTS  effectiveness  as  a  training  device  is 
its  data  collection  function.  It  records  the  state  of  the  system,  trainee 
actions,  and  instructor  inputs  for  later  analysis  and  feedback.  Based  upon 
the  performance  indicators  and  other  relevant  recorded  data,  feedback 
information  and  associated  displays  are  generated  for  presentation  to  the 
trainee  prior  to,  during,  or  following  an  attack  center  exercise. 

While  the  SMARTTS  design  approach  is  to  complement,  rather  than  replace 
the  primary  simulation  of  the  training  devices,  there  are  two  data 
generation  functions  within  SMARTTS  which  provide  simulation.  First  is  the 
Automatic  Interactive  Target  (AIT).  The  AIT  is  a  computer  controlled 
adaptive  target  model.  Its  decisions  and  actions  are  executed  as  a 
consequence  of  the  trainees'  ownship  action,  thus  providing  a  realistic 
simulation  of  enemy  actions.  The  second  simulation  function  consists  of  a 
set  of  models  used  in  the  standalone  mode  to  support  classroom 
investigation.  Although  not  as  sophisticated  as  the  main  simulation  models, 
they  provide  necessary  and  reasonable  simulation  for  alternative  tactical 
analyses  and  scenario  preview  operations. 

SMARTTS  has  been  designed  to  provide  both  standalone  operation  and 
integrated  operation  with  the  simulation  subsystem  of  the  training  device 
( i . e . ,  attack  center).  While  the  specific  information  generated  and 
presented  via  SMARTTS  pertains  to  submarine  tactics  training,  the  basic 
design  of  SMARTTS  is  generic  and  capable  of  being  applied  in  a  variety  of 
training  applications. 

The  SMARTTS  preprototype  provides  capabilities  in  support  of  a  wide  range 
of  instructor  functions,  provides  capabilities  to  assist  the  instructor  in 
achieving  an  effective  trainee  interface  with  the  training  process,  and 
provides  rudimentary  capabilities  to  assist  in  training  system  management. 
These  capabilities  support  the  four  major  functions  of  the  training 
process:  (a)  exercise  development,  (b)  monitor  and  control  of  the  real-time 
training  scenario,  (c)  instructing  the  student,  and  (d)  training  system 
management. 

SMARTTS  represents  the  initial  application  of  TAT  to  an  operational 
training  device.  For  this  reason  it  is  a  preprototype  unit,  and  likely  to 
require  refinement  as  it  is  used.  The  software  structure  is  flexioly 
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designed  to  permit  evolution  (i.e.,  reasonable  modification  of  performance 
indicators,  display  formats,  models,  AIT  actions,  and  so  on).  Furthermore, 
SMARTTS  has  opened  a  major  area  of  development  --  training  assistance 
technology.  Hence,  a  research  and  development  plan  has  been  developed  to 

map  subsequent  TAT  and  SMARTTS  research  and  development  efforts.  The 
Recommendations  Section  (V)  of  this  report  identifies  101  issues  for  further 
investigation  and  development,  associated  with  three  areas  of  work:  (a  ) 
21A41A  Site  Work;  ( b)  TAT  Research  and  Development;  and  (c)  Other  Areas. 

Recommendations  are  also  given  regarding  additional  dimensions  and  aspects 
related  to  each  issue. 

The  next  iteration  of  this  research  and  development  plan  should  key  on 
those  issues  that  have  been  given  the  highest  priority  (i.e.,  number  one). 
Several  of  these  are  associated  with  very  low  cost,  and  hence  should  be 

addressed  immediately.  Others  have  substantially  higher  associated  costs, 
and  should  be  given  priority  in  the  long-term.  It  may  also  be  desirable  to 
address  issues  that  have  a  medium  priority  (i.e.,  number  two)  with 
associated  low  costs,  since  these  are  likely  to  provide  cost/effective 
results  in  the  near-term. 

The  initial  research  and  development  emphasis  forthcoming  from  this  plan 
should  be  focused  on  site  work,  since  it  is  imperative  that  SMARTTS  be 

responsive  to  site  needs  as  originally  designed.  This  will  not  only  improve 
the  cost/effectiveness  of  the  ongoing  operational  training  on  the  21A41A 
SCST,  but  will  also  assist  in  obtaining  meaningful  user  evaluation  of  the 
SMARTTS  preprototype  (i.e.,  the  initial  installation  of  SMARTTS  was  intended 
to  evolve  during  the  test  and  evaluation  period  by  improving  the  Pis, 
feedback  displays,  etc.).  The  training  assistance  technology  addressed  in 
the  SMARTTS  preprototype  and  those  issues  addressed  in  this  research  and 
development  plan  map  out  the  training  subsystem  of  the  training  device.  The 
products  emanating  from  investigative  efforts  in  this  area  will  have 
substantial  impact  on  the  cost/effectiveness  of  all  training  devices  and 
systems. 
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PREFACE 


This  report  is  the  final  document  of  the  SMARTTS  preprototype 
development.  SMARTTS  represents  the  culmination  of  many  years  of  research 
and  development,  resulting  in  the  development  of  a  preprototype  application 
of  Training  Assistance  Technology.  This  preprototype,  although  not  fully 
evaluated,  has  already  ushered  in  a  new  era  in  training  device  design, 
insuring  that  a  training  subsystem  will  be  carefully  considered  in  the 
design  of  every  training  device.  The  SMARTTS  preprototype,  and  its 
effective  implementation,  is  due  to  the  cohesive  efforts  of  the  three  major 
elements  of  the  project  team  --  the  ASW  Training  Center  (user),  the  U.S. 

Naval  Training  Equipment  Center  (procuring  agency),  and  Eclectech 
Associates,  Incorporated  (contractor).  It  is  fitting,  therefore,  that  those 
individuals  having  major  contributions  to  the  accomplishments  of  this 

program  be  recognized. 

The  training  assistance  technology  concepts  embodied  in  the  SMARTTS 
preprototype  are  the  result  of  more  than  a  decade  of  research  and 

development  efforts.  The  initial  conceptual  work  leading  to  the  development 
of  training  assistance  technology  was  achieved  in  the  late  1 960 * s  and  early 
1970's  through  work  with  the  SUBSCOL  staff.  New  London,  Connecticut.  This 
work  has  more  recently  culminated  with  the  development  of  the  preprototype 
SMARTTS  at  the  ASW  Training  Center  in  Norfolk,  Virginia.  Many  individuals 
have  contributed  in  various  ways  to  the  training  assistance  technology 
concepts,  the  various  research  efforts,  and  the  development  of  the  SMARTTS 
preprototype.  It  is  beyond  the  scope  of  this  report  to  document  the 

contributions  of  all  participants.  However,  several  individuals  have  made 
exceptional  contributions  to  the  SMARTTS  preprototype.  These  are  noted 
below. 

The  ASW  Training  Center  in  Norfolk  was  an  essential  element  of  the 
SMARTTS  development.  Their  willingness  to  receive  the  preprototype  system, 
their  assistance  in  providing  us  with  many  useful  recommendations  regarding 
specific  design  aspects,  and  their  substantial  assistance  in  the  actual 
installation  was  invaluable  to  the  project.  Special  acknowledgement  is  due 
LT  R.  Zeller,  and  LT  S.  Ellis,  both  former  department  heads,  submarine 
tactics  training;  LT  H.  O' Bryant,  department  head,  submarine  tactics 
training;  LTjg  R.  Oulin,  department  maintenance  officer;  STSC  Rich,  TDCM  J. 
Wood,  and  STSC  Hafele.  These  individuals  and  their  staff  provided 
exceptional  assistance  to  the  project  throughout  its  duration,  and  of 
greatest  importance,  enthusiastic  support  to  the  use  of  new  technology  to 
improve  submarine  tactics  training. 

The  second  major  element  of  the  SMARTTS  team  was  Code  N71,  U.S.  Naval 
Training  Equipment  Center.  Mr.  William  P.  Lane,  director  of  the  Human 
Factors  Laboratory,  was  a  part  of  the  initial  conceptual  development  of 
Training  Assistance  Technology,  and  a  staunch  fighter  for  its  development 
and  application  to  submarine  tactics  training.  His  tenacity  assured  its 

full  development.  LT  Thomas  N.  Crosby,  SMARTTS  acquisition  director,  was 

the  primary  driving  force  behind  the  preprototype  development.  As  the 

Navy's  representative  he  led  the  SMARTTS  team.  Mr.  W.  Lunceford  was  the 

NTEC  engineering  representative.  He  provided  exceptional  assistance  in 
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helping  to  assure  the  project  met  its  technical  goals.  Dr.  R.  Reynolds 
assisted  LT  Crosby  in  overseeing  the  project,  and  assumed  the  duties  of 
acquisition  director  after  the  latter's  transfer. 

The  third  and  final  major  element  of  the  SMARTTS  team  is  Eclectech 
Associates,  Incorporated,  and  the  many  individuals  who  have  contributed 
substantially  of  themselves  to  the  many  aspects  of  the  project.  Their 
collective  contribution  is  SMARTTS1  The  key  individuals  were  Dr.  G.  Lloyd, 
software  design;  Mr.  F.  Ewalt,  modeling  and  training;  Mr.  M.  Hayes,  hardware 
system  design;  and  Mr.  R.  Friedmann,  site  liaison,  modeling,  and  training. 
These  were  assisted  by  Mr.  S.  Weinstein,  software;  Mr.  0.  Bennett,  software; 
Mrs.  P.  Sherrard,  AIT  software;  Mrs.  P.  Rabe,  software;  Mr.  W.  Henry, 
training;  technical  publications  staff;  Mr.  R.  Gomes,  hardware;  Mrs.  S. 
Shannon,  secretary,  Ms.  J.  Fryer,  secretary;  and  Mrs.  V.  Pittsley,  training. 

SMARTTS  is  the  result  of  a  close  and  strenuous  relationship  between  the 
user,  procuring  agency,  and  contractor. 


NAVTRAEQUIPCEN  80-00079 


TABLE  OF  CONTENTS 


Section  Title  Page 


EXECUTIVE  SUMMARY  1 

PREFACE  5 

I  INTRODUCTION  11 

1.1  Tactics  Training  Background  11 

1.2  TAT  Background  12 

II  DEVELOPMENTAL  APPROACH  17 

2.1  Front-end  Analysis  17 

2.1.1  Tactical  Behavior  Analysis  (Part  1)  19 

2.1.2  Performance  Criteria,  Indicators,  and  Standards  (Part  2)  19 

2.1.3  Training  Structure  and  Process  Development  (Part  3)  20 

2.1.4  Automatic  Interactive  Target  Model  (Part  4)  20 

2.1.5  Hardware  and  Software  Considerations  (Part  5)  21 

2.1.6  Type  A  Specification  (Part  6)  21 

2.1.7  SCST  Sonar  Training/Integration  (Part  7)  22 

2.2  Control  Group  Investigation  22 

2.2.1  Data  Collection  Tool  Development  23 

2.2.2  Data  Collection  24 

2.2.3  Performance  Evaluation  24 

III  SMARTTS  DESCRIPTION  27 

3.1  SMARTTS  Hardware  27 

3.2  SMARTTS  Characteristics  34 

3.2.1  SMARTTS  Training  34 

3. 2. 1.1  Exercise  Preview  34 

3. 2. 1.2  Exercise  Run  35 

3. 2. 1.3  Freeze  36 

3. 2. 1.4  Exercise  Playback  36 

3. 2. 1.5  SMARTTS  Training  Process  36 

3.2.2  Performance  Indicators  38 

3.2.3  Tactical  Variables  40 

3.2.4  Information  Displays  40 

3.2.5  Alternative  Tactics  49 

3.2.6  Instructor  Cues  51 

3.2.7  Exercise  Library  53 

3.2.8  Time  Flag  Units  53 

3.2.9  Automatic  Interactive  Target  53 

3.2.10  Training  Support  Materials  56 

3.3  SMAPTTS  Software  60 

IV  Pr  .-IMINARY  CONCLUSIONS  .  61 


NAVTRAEQUIPCEN  80-00079 


TABLE  OF  CONTENTS  (CONTINUED) 


Section 

Title 

Page 

V 

RECOMMENDATIONS 

63 

5.1 

Research  and  Development  Plan  Introduction 

63 

5.2 

Research  and  Development  Issues 

67 

5.2.1 

21A41A  Site  Work 

67 

5. 2. 1.1 

Preprototype  Test  and  Evaluation 

67 

5. 2. 1.2 

Refine  SMARTTS 

68 

5. 2. 1.3 

Add  TAT  Capabilities 

70 

5. 2. 1.4 

Test  and  Evaluate  New  TAT  Capabilities 

76 

5. 2. 1.5 

SCST  Site  Survey 

76 

5.2.2 

TAT  Research  and  Development 

76 

5. 2. 2.1 

Automatic  Interactive  Target 

77 

5. 2. 2. 2 

Alternative  Actions 

78 

5. 2. 2. 3 

Performance  Indicators 

80 

5. 2. 2.4 

Information  Presentation 

81 

5. 2. 2. 5 

Processing  Support 

86 

5. 2. 2. 6 

Instructor-Specific  Characteristics 

88 

5.2. 2. 7 

Trainee  Specific  Characteristics 

92 

5. 2. 2. 8 

Training  System  Management 

94 

5. 2. 2. 9 

Training  Assistance  Technology  Guidelines 

94 

5.2.3 

Other  Areas 

95 

5. 2. 3.1 

Gaming 

96 

5. 2. 3.2 

Training  Methodologies 

96 

5. 2. 3.3 

Training  Effectiveness  Measurement  and  Evaluation 

96 

5. 2.3.4 

Embedded /Organic  Training 

98 

5.3 

Research  and  Development  Plan  Summary 

99 

BIBLIOGRAPHY 

109 

APPENDICES 


A 

Instructor  Functions 

111 

B 

SMARTTS  Software  Description 

117 

C 

SMARTTS  Reports  Developed  by  Eclectech  Associates,  Inc. 

123 

LIST  OF  ACRONYMS 

125 

,4 


NAVTRAEQUIPCEN  80-C-Q079 


K 


Figure 


LIST  OF  ILLUSTRATIONS 


Title 


Page 


v 


1 

Major  Outputs  of  SMARTTS  Developmental  Effort 

18 

2 

SMARTTS/SCST  Equipment  Diagram 

29 

3 

IRC  Display  Surface 

30 

4 

Display  of  an  Alphanumeric  Listing  of  the  Appropriate 

Available  Input  Commands 

31 

5 

Attack  Center  General  Arrangement 

33 

6 

Geoplot  Display  Format,  With  a  Block  of  Status 

Information 

43 

7 

Geoplot  with  Line-of -Sight 

44 

8 

Tactical  Display  Showing  Target  Status 

45 

9 

Time  Function  Display 

46 

10 

Tri graph  Display 

47 

11 

Alternative  Tactics,  Geoplot  Display 

52 

12 

Overall  AIT  Operation 

55 

13 

The  Training  Process 

57 

14 

Training  Sequence  Chart  -  Preprototype  Training 

59 

15 

Research  and  Development  Plan  Structure 

66 

16 

Conceptual  example  of  three  dimensional  surface 
probability  firing  points  around  target 

of  equal  hit 

83 

17 

Conceptual  example  of  three  dimensional  display 
weapon  deployment 

illustrating 

84 

LIST  OF  TABLES 

Table  Title  Page 

1  Recommendations  Regarding  the  Research  and  Development  Issues  103 


I 

E 

K> 

r.; 

I 


y  ( 


NAVTRAEQUIPCEN  80-00079 


SECTION  I 
INTRODUCTION 

The  Submanjia-Jldvanced  Re act4-ve-  -Tact i cal  -Trcrnrrng-  System  (SMARTTS)  has 
been  "developed  as  the  "training  subsystem"  of  the  Submarine  Combat  System 
Trainer  (SCST),  the  primary  training  device  employed  to  achieve  submarine 
crew  tactical  proficiency.  The  purpose  of  SMARTTS  is  to  provide  a  variety 
of  training  assistance  technology  (TAT)  capabilities  to  the  SCST  to 
substantially  enhance  the  effectiveness  of  the  tactical  training  process. 
SMARTTS  has  been  designed  to  provide  a  range  of  generic  capabilities  which 
can  be  tailored  to  meet  the  needs  of  differing  training  applications.  The 
preprototype  version  of  SMARTTS,  which  is  installed  on  the  21A41A  SCST  at 
the  Fleet  ASW  Training  Center,  Norfolk,  Virginia,  encompasses  a  set  of  TAT 
capabilities  which  have  been  specifically  tailored  to  address  individual 
operator  and  combat  team  training  requirements  associated  with  the  SSN  MK117 
Fire  Control  System.  SMARTTS  is  currently  undergoing  evaluation  in  this 
applied  operational  training  environment. 


1.1  TACTICS  TRAINING  BACKGROUND 


The  U.S.  Navy  Submarine  Force  has  been  experiencing  accelerated 
transition  to  significantly  improved,  although  increasingly  complex,  weapons 
and  tactical  command  and  control  systems  for  both  SSNs  and  SSBNs.  The 
MK116,  MK117,  and  MK118  fire  control  systems,  and  the  AN/BQQ-5  sonar  system 
are  examples  of  the  rapidly  developing  combat  system  capabilities,  bringing 
attendant  increases  in  potential  weapon  system  effectiveness  and  operational 
complexity.  These  technological  advances  are  matched  by  expanding  submarine 
mission  roles  and  tasks  in  tactical  and  strategic  warfare,  and  other 
classified  operations.  The  developing  cruise  missile  vertical  launch 
capability  for  the  SSN  688  class  submarine  is  one  example  of  a  potentially 
substantial  change  in  the  mission  of  the  attack  submarine.  Enhanced 
operational  performance  of  weapons,  fire  control,  sonar,  periscopes, 
electronic  support  measures,  and  other  sensors,  together  with  advanced  ship 
characteristics,  places  heavy  emphasis  on  responsible  tactical  command  and 
control.  High  speed  digital  data  processing,  coupled  with  computer  analysis 
of  multiple  sensor  inputs  requires  highly  developed  operator  visual  and 
interpretive  skills.  These  skills  must  be  supported  by  the  operator's 
ability  to  make  complex  tactical  decisions.  It  comes  as  no  surprise, 
therefore,  that  these  operational  system  developments  have  been  placing 
increasing  demands  on  submarine  tactical  and  operational  training  to  promote 
increased  operational  proficiency  and  readiness. 

The  overall  submarine  tactics  training  system,  whether  providing 
formalized  team  training  in  shore-based  attack  centers  or  individual 
on-the-job  training  at  sea,  principally  exists  to  support  submarine  force 
operational  readiness.  Submarine  combat  system  trainers  for  SSN  and  SSBN 
crews  are  located  at  all  major  submarine  operational  training  centers. 
Simulation  methodology  and  equipment  retrofits  to  these  devices  provide  the 
latest  in  onboard  equipment  developments,  recreating  the  submarine  attack 


11 


PREVIOUS  PAGE 
IS  BLANK 


NAVTRAEQUIPCEN  80-C-0079 


center  and  its  engagement  with  hostile  forces.  Collectively,  these  SCSTs 
represent  the  primary  component  of  the  suomarine  tactical  training  system. 

Submarine  tactics  training  presently  satisfies  a  wide  range  of 
requirements,  from  the  submarine  officer  basic,  indoctrination,  and  advanced 
courses  (SOBC,  SOIC,  and  SOAC),  through  advanced  levels  in  the  prospective 
executive  officer  (PXO)  and  the  submarine  force  commander's  prospective 
commanding  officer  (PCO)  courses.  Training  within  these  courses  spans 
individual  through  team  contexts.  Extensive  refresher  and  predeployment 
training  programs  are  also  conducted  using  the  SCSTs.  Advanced  training 
assistance  technology  is  required  for  these  SCSTs  to  enhance  the  training 
process. 


1.2  TAT  BACKGROUND 


An  instructor  station  has  traditionally  been  a  part  of  every 
simulator/training  device.  However,  it  has  often  been  a  misnomer  consisting 
of  little  more  than  an  operator's  station  for  the  controlling  of  the 
sophisticated  simulator.  Relatively  few  capabilities  have  typically  been 
provided  as  part  of  the  instructor  station  to  enhance  the  instructional 
process  (e.g.,  provide  feedback  to  the  trainees).  The  necessary  technology 
has  existed,  both  in  the  form  of  hardware/software  capabilities  and 
training/instructional  techniques,  for  the  development  highly  cost  effective 
training  tools  integrated  into  the  training  device  to  enhance  the 
instructional  process.  Thus,  a  "bonafied"  training  subsystem  embodying  TAT 
capabilities  can  and  should  be  a  part  of  every  training  device. 


The  General  Accounting  Office,  in  their  report  to  Congress  concerning 
"How  to  Improve  the  Effectiveness  of  U.S.  Forces  Through  Improved  Weapon 
System  Design"  (1981),  focused  on  the  importance  of  the  operator  to  the 
overall  effective  functioning  of  the  weapon  system.  They  estimated  that 
human  errors  account  for  at  least  50  percent  of  the  failures  of  major  weapon 
systems.  They  further  subdivided  these  failures  into  operator  skill  level 
and  proficiency  limitations,  among  other  factors.  Their  findings  attribute 
the  operational  and  readiness  problems  to  the  increasing  complexity  of 
modern  weapon  systems.  An  important  issue  evident  from  this  investigation 
is  the  need  for  improved  training  as  one  means  of  improving  operator 


proficiency  and  reducing  operator  error.  This  points  to  the  need  for  more 
effective  training  systems,  of  which  the  training  subsystem  is  a  major 
contributing  part.  Also,  the  complexity  of  the  training  device  and  training 
system  is  increasing  in  parallel  with  that  of  operational  systems.  Hence, 
more  effective  human  factors  design  of  the  training  system  from  an 
instructional  process  standpoint  is  required.  Both  of  these  findings  point 
to  the  obvious  requirement  to  design  the  training  system  with  particular 
regard  for  the  instructor  and  trainee  interfaces. 

The  problem  of  developing  a  more  effective  training  system  is  not  really 
new,  since  the  training  community  has  always  been  concerned  with  the 
effectiveness  of  instruction.  The  predominant  emphasis  in  the  design  of 
training  devices  and  systems  in  recent  years,  however,  has  been  placed  on 
the  engineering  aspects,  such  as  concern  over  adequate  simulation  fidelity 
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and  the  technology  to  achieve  that  fidelity.  Although  cost-effective 
fidelity  should  be  a  major  issue,  proportionate  emphasis  should  be  given  to 
the  other  training-related  aspects  of  the  training  device  and  system. 
Recent  research  (Hammell,  Gynther,  Grasso,  and  Gaffney,  1981)  has  shown  that 
instructor  characteristics  may  have  a  greater  impact  on  the  effectiveness  of 
training  then  substantial  differences  in  simulator  fidelity.  The  importance 
of  this  finding  is  that  the  instructor  typically  embodies  most  of  those 
nonsimulation  characteristics  of  the  training  device  and  training  system 
(e.g.,  exercise  design,  monitoring  of  student  performance,  and  student 
feedback).  Substantial  gains  in  training  effectiveness,  therefore,  should 
arise  by  providing  TAT  capabilities  that  directly  address  the  training 
process.-  The  training  device,  therefore,  should  be  more  than  just  a 
simulator.  It  should  provide  TAT  capabilities  to  directly  support  the 

instructor  and  trainee  interfaces,  and  hence  directly  support  the  training 
process. 

The  advanced  training  concepts  embodied  in  TAT,  and  those  included  in 
SMARTTS,  directly  stem  from  two  studies  sponsored  by  the  Naval  Training 
Equipment  Center  (Hammell,  Sroka,  and  Allen,  1971;  Hammell,  Gasteyer,  and 
Pesch,  1973).  These  studies  investigated  the  then  current  and  future  needs 
of  submarine  tactics  training  devices.  A  variety  of  recommendations  were 
forthcoming,  addressing  individual  and  team  training  needs  at  basic  through 
advanced  levels.  The  recommendations  addressed  new  training  devices,  as 

well  as  modifications  to  existing  training  devices.  The  21A  series 

submarine  combat  system  trainers  were  in  use  at  the  time,  but  typically  had 
few  capabilities  built-in  to  assist  the  instructor  in  conducting  the 
training  process.  The  recommended  modifications  to  the  SCSTs  included: 

a.  the  need  for  generation  of  objective  performance  indicators  based  on 
the  already  generated  scenario  parameters, 

b.  a  variety  of  feedback  displays  to  provide  information  to  the 
students  concerning  various  aspects  of  the  exercise  scenario  and  the 
performance  indicators, 

c.  an  instructor's  console  that  can  provide  a  range  of  trainee  and 

scenario  monitoring  and  control  capabilities, 

d.  instructor  cues  automatically  generated  by  the  device, 

e.  an  automatic  interactive  target  controlled  by  the  computer  and 
freeing  the  instructor  from  continuous  activity  of  this  type, 

f.  a  long-term  data  analysis  capability  to  continually  evaluate  and 
upgrade  the  effectiveness  of  the  training  process  and  training  system, 

g.  an  alternative  tactics  capability  to  enable  rapid  investigation  of 
alternative  ownship  and  target  actions  in  the  classroom. 

The  SMARTTS  preprototype  operating  today  is  largely  made  up  of  generic 
training  assistance  technology  capabilities  that  were  recommended  as  part  of 
these  earlier  investigations,  and  refined  and  tailored  to  the  specific 
application  of  the  preprototype. 
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These  early  requirement  studies  were  followed-up  by  a  series  of  research 
and  development  efforts  sponsored  by  the  Naval  Training  Equipment  Center  to 
develop  a  laboratory  subset  of  TAT  capabilities  that  could  be  evaluated,  to 
conduct  laboratory  evaluations  of  the  TAT  training  effectiveness,  and  to 
develop  a  plan  for  the  eventual  installation  of  a  laboratory  training 
subsystem  integrated  with  one  of  the  submarine  combat  system  trainers 
(Pesch,  Hammell,  and  Ewalt,  1974;  Hammell,  Pesch,  Ewalt  and  Rabe,  1976; 
Hammell,  Manning  and  Ewalt,  1979).  As  a  part  of  these  investigations  a 
laboratory  version  of  a  subset  of  the  MK81  Weapons  Control  Console  ( i . e . ,  a 
major  operator  interface  unit  in  the  MK117  fire  control  system)  was 
developed,  with  associated  simulation  models  (e.g.,  ownship  and  target 
maneuvering  characteristics,  ownship  and  target  sensor  characteristics, 
environmental  characteristics)  and  several  major  TAT  capabilities  (e.g., 
performance  indicators,  feedback  displays).  The  potential  of  TAT  to 
substantially  improve  the  effectiveness  of  the  submarine  tactics  training 
process  was  indicated  in  a  laboratory  investigation  with  this  device, 
training  active  duty  submarine  officers  from  Submarine  Group  Two  and 
SUBSCOl.  The  intention,  as  a  result  of  this  empirical  investigation,  was  to 
install  the  1 aboratory-type  TAT  onto  an  existing  SCST  and  evaluate  their 
training  effectiveness  in  the  operational  training  environment. 

As  a  result  of  the  earlier  requirement  studies  and  this  series  of 
laboratory  investigations,  the  need  for  TAT  to  support  the  complex  submarine 
tactics  training  process,  and  the  ready  availability  of  technology  to 
achieve  these  capabilities,  became  evident  to  many  in  the  submarine  tactics 
training  community.  Callan,  Kelly,  and  Nicotra  (1978)  integrated  several 
performance  indicators  and  feedback  displays  into  the  21A40  SCST  in  San 
Diego.  Their  empirical  investigation  in  applied  submarine  tactics  training 
verified  the  need  for,  and  potential  effectiveness  of,  TAT  for  submarine 
tactics  training  integrated  with  the  SCSTs.  The  Submarine  Advanced  Reactive 
Tactical  Training  System  (SMARTTS)  program  emerged  as  a  result  of  these 
investigations  indicating  substantial  training  effectiveness  gains  resulting 
from  TAT,  the  ready  availability  of  hardware/software  to  achieve  the  TAT 
capabilities,  and  the  support  shown  by  the  submarine  tactics  training 
community. 

The  SMARTTS  program  was  initiated  to  develop  a  comprehensive 
preprototype  SMARTTS  training  subsystem  as  a  part  of  an  existing  SCST,  for 
the  purpose  of  evaluating  the  effectiveness  of  TAT  in  operational  submarine 
officer  tactics  training.  The  SMARTTS  preprototype  implementation  was 
directed  specifically  at  training  the  fire  control  party  of  an  SSN  688  class 
fast  attack  submarine,  using  the  MK117  fire  control  system.  The  program 
began  with  an  extensive  front-end  analysis,  extending  the  earlier  determined 
training  requirements  and  research  findings  to  the  development  of  a  set  of 
specific  training  device  and  training  system  characteristics  to  support  the 
objective  training.  This  front-er.d  analysis  laid  out  a  set  of  specific  TAT 
characteristics  recommended  in  support  of  submarine  officer  tactics  training 
utilizing  the  SCST.  The  potentially  most  effective  characteristics 
recommended  by  this  analysis  were  then  designed  as  the  preprototype  version 
of  SMARTTS  for  installation  in  the  21A41  SCST  at  the  Fleet  ASW  Training 
Center  in  Norfolk,  Virginia  (note,  the  21A41  SCST  with  SMARTTS  is  designated 
the  21A41A).  The  preprototype  SMARTTS  was  developed  and  installed,  and  is 
currently  being  used  in  submarine  officer  tactics  training.  The 
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effectiveness  contribution  of  SMARTTS  to  submarine  officer  tactics  training 
is  currently  being  evaluated  by  the  Navy.  Preliminary  evaluation  by  the 
Navy's  submarine  tactics  training  fleet  project  team  has  indicated  approval 
and  support  for  the  preprototype  SMARTTS  TAT  capabilities.  The  final 
evaluation  of  SMARTTS  is  expected  to  occur  after  it  has  been  in  operation 
for  6  months  to  1  year. 

The  remainder  of  this  report  summarizes  the  developmental  approach  for 
implementing  the  SMARTTS  preprototype;  provides  an  overview  of  the  SMARTTS 
hardware,  software,  and  training  characteristics;  and  presents  a 
comprehensive  research  and  development  plan  recommending  work  to  follow  the 
preprotoype  SMARTTS  installation  of  TAT.  It  is  essential  that  the 
preprototype  SMARTTS  be  modified  as  it  is  used  and  evaluated,  to  improve  the 
training  effectiveness  value  of  its  characteristics  and  enhance  training. 
The  continual  evolution  of  the  SMARTTS  characteristics  is  a  fundamental 
concept  in  its  design,  and  necessary  to  insure  substantial  training  gains  as 
training  needs  and  tools  change. 
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SECTION  II 

DEVELOPMENTAL  APPROACH 

A  systematic  design  approach  was  followed  in  the  development  of  the 
SMARTTS  preprototype,  similar  to  that  of  the  Instructional  Systems 
Development  (ISO),  Systems  Approach  to  Training  (SAT),  and  Training  System 
Analysis  (TSA)  approaches.  It  began  with  an  extensive  front-end  analysis 
establishing  the  individual  and  team  behavior  of  members  of  the  fire  control 
party,  identifying  training  objectives,  and  resulting  in  the  identification 
of  specific  training  system  characteristics.  The  resulting  "Type  A 
Specification"  (i.e.,  functional  specification)  formed  the  basis  for  the 
development  of  the  specific  SMARTTS  preprototype  features.  The  hardware  and 
software  development  of  the  preprototype  unit  followed,  with  SMARTTS 
installed  on  the  21A41A  submarine  combat  systems  trainer  in  Norfolk, 
Virginia.  Prior  to  installation  of  SMARTTS,  a  plan  was  developed  for  the 
training  effectiveness  evaluation  of  the  SMARTTS  preprototype.  Control 
group  data  was  collected  on  the  21A41  SCST  as  part  of  the  planned  evaluation 
prior  to  beginning  the  installation  of  SMARTTS.  The  control  group  data 
provided  information  relevant  to  the  effectiveness  of  the  traditional 
training  device  (i.e.,  without  SMARTTS).  The  plan  was,  although  not  yet 
carried  out,  to  empirically  evaluate  the  training  effectiveness  of  SMARTTS 
by  comparing  the  control  group  performance  resulting  from  traditional 
training  with  the  performance  of  crews  trained  under  the  SMARTTS-modif ied 
system. 

The  front-end  analysis  and  the  control  group  data  collection  efforts  are 
summarized  in  the  remainder  of  this  section.  The  SMARTTS  preprototype 
developmental  effort,  in  terms  of  the  approach  to  development  of  the 
hardware,  software  and  training  materials,  is  not  addressed  in  this  report. 
This  developmental  effort  followed  a  standard  engineering  design  and 
manufacturing  approach.  Details  relating  to  the  phases  of  this  development 
are  contained  in  the  various  documents  generated  throughout  the  effort  (see 
Appendix  C).  A  summary  of  the  preprototype  SMARTTS  characteristics, 
however,  are  presented  in  the  next  section  of  this  report  (Section  3, 
SMARTTS  Description). 


2.1  FRONT-END  ANALYSIS 


The  front-end  analysis  conducted  to  identify  the  SMARTTS  TAT 
characteristics  was  traditional  in  that  it  followed  a  systematic  approach 
beginning  with  the  identif ication  of  trainee  behavior,  and  leading  to  the 
identification  of  specific  functional  characteristics  of  the  training 
system.  Additionally,  the  front-end  analysis  investigated  alternative 
hardware  and  software  approaches  prior  to  arriving  at  the  recommended 
SMARTTS  characteristics.  The  major  outputs  of  the  front-end  analysis  are 
summarized  in  Figure  1.  This  figure  shows  the  seven  major  parts  of  the 
front-end  analysis,  associated  with  the  major  outputs.  Each  of  these  parts 
is  summarized  below. 
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2.1.1  TACTICAL  BEHAVIOR  ANALYSIS  (PART  1).  An  extensive  behavioral  data 
base  relating  to  the  tasks  of  individuals  within  the  submarine  combat  team 
(i.e.,  fire  control  party  and  sonar  party)  and  the  team  itself  is  resident 
at  Eclectech  Associates,  Incorporated  as  a  result  of  various  analyses 
conducted  pertaining  to  submarine  officer  tactics  operations  (e.g.,  Hammell 
and  Mara,  1970;  Hammell  et  al.  1971;  Hammell  et  al.  1973).  The  behavioral 
analysis  utilized  the  task,  knowledge  and  skill  information  resident  in  this 
data  base,  updating  it  as  necessary  to  cover  the  more  recent  weapon  system 
developments  (e.g.,  tasks  specifically  related  to  Tomahawk  and  Harpoon 
missiles).  The  major  output  from  this  analysis  was  the  development  of  a 
comprehensive  set  of  behavioral  training  objectives  relating  to  members  of 
the  submarine  fire  control  party. 

It  should  be  noted  that  the  SMARTTS  preprototype  primarily  addresses 
training  of  the  fire  control  party,  and  does  not  specifically  address  the 
sonar  party.  However,  since  the  fire  control  and  sonar  parties  are  closely 
integrated  within  the  combat  team  from  an  operational  standpoint,  and  often 
also  during  training  exercises,  the  potential  applicability  of  SMARTTS 
characteristics  to  the  sonar  party  was  also  investigated.  The  sonar  party 
requirements  were  investigated  under  Part  7  of  the  front-end  analysis, 
separate  from  the  major  thrust  which  focused  on  the  fire  control  party  (Part 
7  is  summarized  later). 

The  major  accomplishments  of  Part  1  were: 

a.  Determined  trainee  entry  characteristics  for  members  of  the 
submarine  fire  control  party,  as  related  to  team  training  involving  the  SCST. 

b.  Identified  terminal  skill  and  knowledge  requirements  for  the 
submarine  fire  control  party,  as  related  to  tactical  team  training  employing 
the  SCST. 

c.  Determined  U.S.  Navy  submarine  missions,  offensive  roles,  and 
operational  tasks,  and  identified  the  preceived  criticality  of  each. 

d.  Identified  and  integrated  into  the  other  accomplishments 
consideration  for  current  and  potential  submarine  tactical  doctrine, 
including  the  characteristics  of  present  and  planned  tactical  hardware 
systems. 


e.  Delineated  the  current  SCST  training-related  capabilities. 

f.  Developed  a  comprehensive  set  of  submarine  fire  control  party 
tactical  training  objectives  for  a  specific  application  in  SSN/SSBN 
refresher  training  type  employment  of  the  SCST. 


2.1.2  PERFORMANCE  CRITERIA,  INDICATORS,  AND  STANDARDS  (PART  2).  The  tasks 
under  this  part  were  directed  toward  the  identification  of  candidate 
performance  indicators  for  inclusion  in  the  preprototype  SMARTTS.  Both 
objective  (i.e.,  automatically  calculated  by  the  computer)  and  subjective 
(i.e.,  unable  to  be  calculated  by  the  computer;  observed  and  estimated  by 
the  instructor)  performance  indicators  were  considered.  Associated  with  the 
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performance  indicators  and  considered  in  this  grouping  were  tactical 
variables  relevant  to  the  performance  of  the  fire  control  party  (e.g., 
signal  to  noise  ratio),  and  warranted  for  provision  of  feedback  information 
during  training.  The  accomplishments  as  follows: 

a.  Developed  a  set  of  comprehensive  submarine  tactical  performance 
criteria  specifically  keyed  to  USN  submarine  missions,  roles  and  tasks. 

b.  Developed  and  verified  submarine  tactical  performance  indicators 
suitable  for  use  in  the  SCST  environment  on  the  basis  of  the  identified 
performance  criteria  and  current  submarine  tactical  doctrine;  relevant 
tactical  variables  were  also  identified. 

c.  Developed  a  plan  of  approach  for  the  use  of  information  generated  by 
performance  indicators  and  tactical  variables  during  the  SCST  training 
process. 


2.1.3  TRAINING  STRUCTURE  AND  PROCESS  DEVELOPMENT  (PART  3).  The  tasks  in 
this  part  continued  the  analysis  begun  during  the  two  earlier  parts  and 
extended  it  to  the  instructor.  The  instructor  task  analysis  yielded 
valuable  information  pertaining  to  required  training  device  and  system 
characteristics  to  support  his  functions.  The  results  of  these  analyses 
were  then  fitted  into  a  macro  and  micro  training  structure,  with  the  former 
addressing  intersite  aspects  and  the  latter  addressing  intrasite  aspects. 
The  accomplishments  were: 

a.  Examined  and  developed  requirements  pertaining  to  the  instructor 
interface  within  the  training  device/system;  this  was  based  on  an  instructor 
task  analysis. 

b.  Examined  and  developed  requirements  pertaining  to  the  trainee 
interface  of  the  training  device/system,  primarily  as  controlled  by  the 
instructor  (e.g.,  feedback  information  requirements). 

c.  Developed  candidate  training  process  functional  characteristics  to 
satisfy  the  instructor  and  trainee  interface  requirements. 

d.  Developed  a  macro  structure  training  management  process  addressing 
training-related  functions  of  the  major  naval  activities  as  they  impact  the 
submarine  tactics  training  system,  and  communication  and  coordination 
between  these  activities. 

e.  Developed  a  micro  structure  training  process  addressing  the 
activities  to  be  performed  locally  at  each  SCST  facility;  this  is  primarily 
concerned  with  the  development  of  instructional  materials  and  the  conduct  of 
the  training  process  at  each  site. 


2.1.4  AUTOMATIC  INTERACTIVE  TARGET  MODEL  (PART  4).  A  major  characteristic 
identified  by  earlier  investigations  (Hammell  et  al.,  1973),  and  receiving 
special  consideration  during  this  analysis  was  the  automatic  interactive 
target,  which  is  a  computer  driven  target  that  reacts  in  a  realistic  manner 
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to  ownship  actions  and  other  factors  in  the  environment.  The  automatic 
interactive  target  (AIT),  in  addition  to  issues  of  relevancy  during  the 
training  process,  required  analysis  regarding  the  feasibility  of  design  and 
application.  The  major  accomplishments  of  this  part  were  as  follows: 

a.  Established  general  tactical  characteristics  of  known  and  expected 
threat  targets  appropriate  for  SCST  training. 

b.  Correlated  expert  operational  experience  with  the  most  recent  source 
documentation  to  establish  a  framework  for  threat  tactical  decisionmaking 
appropriate  to  each  main  threat  platform  group. 

c.  Developed  a  complete  set  of  tactical  action-reaction  sequence 
diagrams  to  cover  known  and  expected  tactical  employment  of  a  high  priority 
threat  submarine  platform  (Type  II  SSGN);  this  platform  was  selected  as  the 
initial  AIT  model. 

d.  Developed  tactical  maneuvering  and  tactical  interaction  response 
criteria  most  likely  to  be  encountered  from  a  high  priority  threat  submarine 
platform  (Type  II  SSGN). 

e.  Identified  AIT  hardware  and  software  functions. 


2.1.5  HARDWARE  AND  SOFTWARE  CONSIDERATIONS  (PART  5).  Alternative  hardware 
and  software  approaches  were  investigated  with  regard  to  accomplishing  the 
functional  characteristics  identified  in  Part  3,  and  the  AIT  model 
identified  in  Part  4.  The  intention  of  these  tasks  was  to  generate 
information  pertaining  to  alternative  hardware/software  approaches  to  the 
design  of  the  SMARTTS  preprototype,  and  the  recommendation  of  a  particular 
design  approach.  The  accomplishments  were  as  follows: 

a.  Summarized  existing  and  projected  SCST  designs. 

b.  Summarized  other  selected  automated  submarine  operational  training 
systems  and  their  hardware/software  technology. 

c.  Developed  a  preprototype  SMARTTS  system  design  approach. 

d.  Identified  alternative  system  conf igurations. 

e.  Conducted  a  system  trade-off  analysis. 

f.  Determined  hardware  and  software  functional  requirements. 


2.1.6  TYPE  A  SPECIFICATION  (PART  6).  The  "Type  A  Specification",  which 
identified  the  functional  character istics  recommended  for  SMARTTS,  was 
developed  under  this  part.  The  accomplishments  were: 

a.  Developed  the  "Type  A  Specification"  identifying  the  SMARTTS  TAT 
characteristics  on  a  functional  level. 
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b.  Developed  a  training  plan  and  schedule  for  implementation  of  the 
preprototype  SMARTTS. 


2.1.7  SCST  SONAR  TRAINING/INTEGRATION  (PART  7).  The  applicability  of  the 
SMARTTS  characteristics  to  sonar  training  was  investigated  under  this  part. 
The  final  report  for  the  front-end  analysis  was  also  developed  under  this 
part.  The  accomplishments  were: 

a.  Analysis  of  sonar  operational  team  training  requirements  with  regard 
to  the  joint  operation  mode  of  the  SCST  and  the  sonar  trainer  (e.g.,  21B64). 

b.  Expanded  the  Type  A  Specification  to  include  sonar  operational  team 
training  considerations. 

c.  Identified  site-specific  functional  requirements  for  the  21A41  SCST, 
which  would  employ  the  preprototype  SMARTTS  installation. 

d.  Development  of  a  top-level  system  definition  specification  for  the 
preprototype  installation  at -the  21A41  SCST  site. 

e.  Development  of  the  front-end  analysis  final  report  (Hammell,  Henry, 
Ewalt,  Natter  and  Chidley,  1980). 

The  above  discussion  indicates  the  products  developed  during  the 
front-end  analysis,  as  well  as  provides  an  indication  of  the  approach  taken 
in  that  analysis.  The  final  report  of  this  analysis  provides  greater  detail 
of  the  methodology  and  results.  It  also  summarizes  observations  of  SCST 
training  at  the  various  sites;  provides  substantial  information  regarding 
alternative  training  methodologies,  instructor  functions,  guidance  relevant 
to  the  setup  and  conduct  of  the  training  process  (e.g.,  scenario  resign, 
postscenario  briefing),  operator  and  instructor  training  requirements  and 
recommendations  in  support  of  the  SMARTTS  subsystem,  and  the  design  and 
application  of  the  recommended  SMARTTS  characteristics. 


2.2  CONTROL  GROUP  INVESTIGATION 


The  preprototype  SMARTTS  development  and  installation  on  the  21A41A  SCST 
was  a  research  and  development  effort  to  evaluate  the  training  effectiveness 
resulting  from  the  integration  of  TAT  into  an  existing  training  device  in  an 
applied  operational  training  setting.  A  necessary  part  of  the  SMARTTS 
effort,  therefore,  is  the  empirical  evaluation  of  the  change  in  the 
effectiveness  of  submarine  officer  tactics  training  resulting  from  the 
addition  of  SMARTTS.  An  experimental  design  was  developed  for  the  empirical 
investigation,  to  generate  data  pertaining  to  the  effectiveness  of  tactics 
training  prior  to  and  following  the  installation  of  SMARTTS  on  the  21A41A 
SCST,  and  to  yield  statistically  supportable  results  addressing  the  training 
effectiveness  of  specific  SMARTTS  TAT  capabilities.  The  experimental  design 
focused  on  a  comparison  of  the  training  gain  achieved  during  a  standard 
tactics  training  course  using  the  traditional  21A41  SCST,  with  the  training 
gain  achieved  during  the  same  course  using  the  integrated  SMARTTS/21A41A 
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SCST  device.  This  comparison  would  demonstrate  the  change  in  training 
effectiveness  due  to  the  installation  of  SMARTTS. 

Submarine  tactical  operations  is  a  complex  process  involving  a  team  of 
officers  and  enlisted  personnel.  The  fire  control  party  typically  has  12  to 
15  members,  each  of  whom  performs  different  functions,  and  interacts  with 
different  operational  devices.  The  measurement  of  individual  and  overall 
team  performance,  therefore,  is  inherently  complex  and  difficult.  Although 
objective  performance  indicators  are  available,  such  as  those  developed  and 
implemented  under  SMARTTS,  they  typically  do  not  address  the  overall 
performance  of  the  team.  Rather,  they  address  team  and/or  individual 
performance  pertaining  to  a  specific  subset  of  tactical  concerns  (e.g., 
probability  of  counterdetection  addresses  one  subset  of  concern  during  a 
tactical  encounter).  The  various  performance  measures  addressing  different 
issues,  which  may  change  in  conflict  with  each  other  as  a  function  of 
ownship  operations,  are  extremely  useful  from  a  training  standpoint.  Their 
purpose  is  to  provide  information  regarding  the  relevant  and  often  diverse 
issues.  However,  assessment  of  training  effectiveness  necessitates  cohesive 
measures  which  summarize  the  team's  performance.  Also,  many  of  the  relevant 
aspects  of  tactical  performance  are  not  readily  observable  in  computer  data, 
but  must  be  evaluated  by  a  trained  observer.  Hence,  a  multifaceted 
evaluation  tool  was  necessary  to  address  (a)  overall  team  performance,  (b) 
different  aspects  pertaining  to  subsets  of  team  and  individual  performance, 
and  (c)  objective  and  subjective  evaluation  data. 

Several  data  collection  and  evaluation  tools  were  developed  with  which 
to  observe  training  and  generate  data  pertaining  to  the  level  of  tactical 
team  performance.  The  tools  focused  on  the  collection  of  specific  tactical 
parameter  data,  and  observation  by  experienced  submarine  personnel  of 
specific  aspects  of  team  performance  (e.g.,  communication).  The  data 
collection  tools  and  techniques  were  evaluated  in  a  "pilot"  evaluation, 
during  which  submarine  teams  were  actually  observed  in  training.  The  tools 
were  subsequently  modified  on  the  basis  of  this  preliminary  evaluation. 


2.2.1  DATA  COLLECTION  TOOL  DEVELOPMENT.  The  data  collection  and  evaluation 
tools  were  specifically  developed  for  SSN  688  class  fire  control  team 
members,  21A41  SCST  instructor  performance,  and  for  use  in  the  SMARTTS 
evaluation.  The  tools  were  developed  to  encompass  refresher/postoverhaul , 
predeployment,  and  TRA-time  trainer  employment.  They  were  prepared  to 
discover  a  broad  scope  of  fire  control  team  performance  factors  including 
those  of  the  fire  control/sonar  plotters,  MK81  operators,  fire  control 
coordinator,  and  approach  officer  utilizing  the  MK 1 1 7  fire  control  system. 
The  evaluation  tools  were  organized  to  be  compatible  with  both  the  current 
21A41  configuration  and  the  future  SMARTTS  21A41A  preprototype 
installation.  The  evaluation  tools  also  reflected  the  requirements  of  the 
submarine  tactical  doctrine  and  current  21A41  staff  concepts  of  trainee 
performance  and  training  difficulty.  The  data  collection  format  and 
evaluation  criteria  were  developed  working  in  close  liaison  with  the  21A41 
SCST  staff. 


Information  considered  for  inclusion  in  the  data  collection  tools  were: 
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a.  Crew  background  and  portion  of  tactical  course  that  was  applicable. 

b.  Applicable  exercise  training  issues  and  overall  objectives  as 
identified  by  the  instructor  and/or  crew  commanding  officer. 

c.  Applicable  21A41  SCST  tactics  course  --  operational  objective  as 
identified  by  the  instructor,  squadron  training  officer  and/or  commanding 
officer. 

d.  Exercise  difficulty  level,  number  and  identification  of  targets,  mix 
of  threat  versus  nonthreat  targets. 

e.  Scenario  initial  setup  and  background  conditions,  geometry, 
environment,  use  of  sensors,  etc. 

f.  Target  actions,  maneuvers,  and  times  of  occurrence. 

g.  Ownship  actions,  maneuvers,  and  times  of  occurrence. 

h.  Weapon  employment  --  type,  number,  time  of  fire,  initial  settings, 
postfire  changes  and  settings,  torpedo  run,  and  hit/miss. 

i.  Individual  and  team  level  performance  to  prescribed  criteria. 

The  data  collection  effort  used  five  data  forms  consisting  of  the 

following: 

Part  A:  Overall  evaluation  of  team  work. 

Part  B:  Instructional  format,  regarding  the  tasks  of  instruction. 

Part  C:  Fire  control  party  individual  training  evaluation,  addressing 

the  performance  of  each  operator. 

Part  D:  Summary  of  exercise  sequence,  evaluating  team  performance 
factors  during  each  sequence  of  the  exercise. 

Part  E:  Fire  control  party  individual  training  evaluation,  evaluating 

the  improvement  of  performance  over  the  course  for  each  trainee. 

2.2.2  DATA  COLLECTION.  Data  were  collected  over  a  6-month  period, 

observing  and  evaluating  the  training  performance  of  teams  on  the  21A41  SCST 
in  Norfolk,  and  the  21A37  SCST  in  New  London.  The  data  collection  was 
accomplished  at  these  sites  prior  to  initiation  of  extensive  liaison  between 
the  21A41  site  personnel  and  the  SMARTTS  project  team.  Since  many  of  the 
relevant  SMARTTS  characteristics  are  manually  implemented  without  a 
computer-based  capability,  it  was  believed  that  interaction  between  the 
SMARTTS  team  and  site  personnel  would  have  a  positive  impact  on  the 
effectiveness  of  the  training  being  conducted.  Since  the  control  group  data 
represent  the  traditional  training  effectiveness  without  SMARTTS,  it  was 
necessary  to  collect  those  data  prior  to  the  site  having  interaction  with 
the  SMARTTS  project  team. 

2.2.3  PERFORMANCE  EVALUATION.  The  data  collected  from  the  three  training 
groups  (i.e.,  refresher/postoverhaul ,  TRA-time,  and  PCO  training)  will  be 
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directly  compared  to  comparable  groups  observed  in  training  after  tne 
installation  of  SMARTTS.  The  observations  to  be  made  after  the  SMARTTS 
installation  were  planned  to  use  the  same  forms  and  data  collection 
procedures  used  to  obtain  the  control  group  data.  The  planned  comparisons 
address:  (a)  overall  team  performance,  (b)  team  work  improvement  over 
exercise  sequences,  and  (c)  instructor  performance.  It  should  be  noted, 
that  information  regarding  the  instructor's  tasks  and  functioning  is  also  a 
part  of  the  data  collection,  since  SMARTTS  is  aimed  at  assisting  the 
instructor  in  performing  his  functions.  The  MK 117  fire  control  system 
combat  team  will  be  evaluated  in  tactical  areas  including  information 
generation,  information  use,  sequencing  actions,  communications, 
coordination  involving  interpersonnel  behaviors,  and  attitude. 

Only  the  control  group  data  have  been  collected  at  the  current  time. 
These  data  become  meaningful  only  when  compared  with  those  collected  during 
SMARTTS/21A41A  SCST  training.  The  post-SMARTTS  training  effectiveness  data 
collection  effort  and  the  subsequent  evaluation  of  the  training 
effectiveness  contribution  made  by  SMARTTS  is  under  consideration  by  the 
Navy.  It  is  uncertain  at  this  time  whether  this  test  and  evaluation  will 
occur,  and  if  so  to  what  extent. 

The  detailed  information  concerning  the  SMARTTS  control  group  data 
collection  effort  is  contained  in  the  final  report  of  that  task  (Ewalt, 
Hammell,  Henry  and  Natter,  1981). 
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SECTION  III 
SMARTTS  DESCRIPTION 


SMARTTS  is  a  preprototype  strap-on  unit  to  the  21A41  Submarine  Combat 
System  Trainer  (SCST)  at  the  Fleet  ASW  Training  Center,  Atlantic  located  in 
Norfolk,  Virginia  (note,  the  new  trainer  designation  with  SMARTTS  is 
21A41A).  The  SMARTTS  preprototype  is  capable  of  either  standalone 
operation,  or  operation  integrated  with  the  simulation  subsystem  of  the 
21A41  SCST  in  support  of  attack  center  training  and  classroom  training.  The 
21A41A  SCST,  including  the  SMARTTS  addition,  addresses  the  training  of 
combat  teams  of  the  SSN  688  class  submarine,  using  the  MK117  fire  control 
system  and  AN/BQQ5  sonar.  SMARTTS  hardware  consists  of  an  instructor 
console  located  in  the  attack  center  to  monitor  and  control  various  aspects 
of  the  problem,  monitor  aspects  of  trainee  performance,  and  control 
information  to  be  presented  on  two  overhead  CRT  displays  in  the  attack 
center.  A  similar  instructor  console  is  located  in  the  adjacent  classroom 
to  control  a  large  screen  display.  Both  the  classroom  and  attack  center 
subsystems  can  be  operated  prior  to,  during,  and  following  an  attack  center 
exercise.  The  intention  of  this  design  is  to  support  a  variety  of  tactics 
training  needs  across  multiple  courses  and  levels  of  trainees. 

The  essence  of  SMARTTS  is  its  capability  to  collect  and  generate  a 
variety  of  information  pertinent  to  the  tactical  training  problem,  and  to 
display  various  combinations  of  that  information  on  the  instructor  console, 
the  classroom  large  screen  display  and  the  attack  center  overhead  monitors 
at  appropriate  times  during  the  training  process.  Substantial  su  -port 
capabilities  are  resident  in  the  system  to  manipulate  the  information  in 
various  ways  as  required  during  the  training  process. 

The  preprototype  SMARTTS  does  not  incorporate  all  of  the  characteristics 
recommended  in  the  "Type  A  Specification"  of  the  front-end  analysis. 
Rather,  the  preprototype  unit  includes  those  characteristics  deemed  to  be 
essential  to  the  SMARTTS  concept  of  TAT,  and  which  were  feasible  within 
budget  and  time  constraints.  Characteristics  beyond  those  incorporated  in 
the  preprototype  unit  (e.g.,  instructor  interface  language)  should  be 
carefully  considered  in  future  developments.  Section  5,  Recommendations, 
provides  a  research  and  development  plan  identifying  areas  for  further 
development  of  TAT  following  from  the  SMARTTS  preprototype.  Several  of 
these  are  concepts  put  forth  in  the  "Type  A  Specification",  but  not  included 
in  the  preprototype  unit. 

A  summary  overview  of  the  SMARTTS  system  is  provided  in  the  remainder  of 
this  section.  This  includes  (a)  a  hardware  overview  identifying  the  major 
hardware  components  of  SMARTTS,  (b)  a  description  of  the  SMARTTS 
characteristics,  and  (c)  a  brief  summary  description  of  the  SMARTTS  software. 


3.1  SMARTTS  HARDWARE 


The  essence  of  SMARTTS  is  in  the  training-related  capabilities  that  are 
implemented  via  hardware,  software,  and  training  materials.  The  additional 
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hardware  added  to  the  21 A41  SCST  to  support  SMARTTS  was  not  substantial.  In 
fact,  the  SMARTTS  capabilities  could  be  (i.e.,  and  should  be)  integrated 
into  training  devices  at  the  time  of  initial  concept  design;  and  followed 
through  their  developmental  process.  Many  of  these  SMARTTS-type 
capabilities  have  their  roots  in  processes  that  are  already  carried-on 
inside  typical  computer-based  training  devices.  If  incorporated  in  the 
initial  design,  the  SMARTTS  unit  would  comprise  the  instructional  subsystem, 
simply  interfaced  with  the  simulation  subsystem. 

The  SMARTTS  hardware  is  divided  into  three  major  groups,  (a)  the  attack 
center  subsystem,  (b)  the  classroom  subsystem,  and  (c)  the  model  deck 
subsystem.  The  model  deck  subsystem  consists  of  the  host  computer  (VAX 
11/780)  for  SMARTTS,  with  appropriate  interfaces  to  the  MK117  fire  control 
system,  the  21A41A  SCST  simulation  computer,  and  other  necessary  devices. 
Virtually  all  of  the  SMARTTS  data  collection  and  processing  is  conducted  by 
the  host  computer.  The  major  components  of  SMARTTS  hardware,  integrated 
with  the  existing  21A41A  SCST  hardware,  are  illustrated  on  the  component 
block  diagram  of  Figure  2.  They  are: 

a.  VAX  11/780  SMARTTS  host  computer  (left  of  center  in  diagram).  This 
host  computer  is  connected  to  all  of  the  other  SMARTTS  components,  and 
functions  to  collect  data,  process  the  data,  and  drive  the  various 
information  displays. 

b.  Interface  to  the  main  simulation  computer  group  (MSCG,  UYK-7) 
(center  of  Figure  2).  This  is  the  two-way  interface  between  the  SMARTTS 
host  computer  and  the  21A41A  SCST  simulation  computer  (i.e.,  a  computer  that 
simulates  targets,  the  environment,  etc.). 

c.  Interface  between  the  SMARTTS  host  computer  and  the  MK117  fire 
control  system  computer  (UYK-7)  (upper  right-hand  of  Figure  2).  This  is  the 
interface  to  the  operational  fire  control  system  computer.  Information  is 
collected  from  the  MK117  regarding  various  aspects  of  the  ongoing  tactical 
problem. 

d.  Instructor  Remote  Console  (IRC)  (near  upper  center  of  Figure  2). 
The  instructor  remote  console  is  the  instructor's  console  located  in  the 
attack  center.  The  IRC  has  two  display  surfaces  (see  Figure  3):  the  upper 
surface  is  a  color  graphic  CRT,  and  the  lower  surface  is  a  touch  sensitive 
plasma  display.  The  upper  CRT  is  used  to  present  a  variety  of  information 
to  the  instructor  in  both  graphical  and  alphanumeric  formats.  The 
information  display  formats  discussed  later  in  this  section  (3.2.4)  provide 
examples  of  information  presented  on  the  upper  display.  The  lower  plasma 
display  is  used  as  the  primary  instructor  entry  device.  It  displays  an 
alphanumeric  listing  of  the  appropriate  available  input  commands  at  any 
point  in  time  (see  Figure  4).  It  has  an  XY  touch  sensitive  grid  pattern 
that  permits  the  instructor  to  point  to  his  choice  of  input  commands,  ana 
thus  enter  commands  into  the  system.  The  lower  plasma  display  also  presents 
some  status  information.  The  IRC  can  be  used  to  investigate  various 
scenarios  and  their  pertinent  parameters  prior  to  running  them  in  real-time 
on  the  trainer,  to  setup  an  exercise  on  the  trainer,  to  control  various 
activities  during  a  training  exercise  (e.g.,  maneuver  targets),  to  monitor  a 
wide  variety  of  performance  indicators  and  tactical  information  during  the 
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training  exercise,  and  to  select  and  control  information  for  presentation  on 
the  overhead  displays  in  the  attack  center,  either  prior  to,  during  or 
following  the  actual  real-time  exercise  in  the  attack  center. 

e.  The  two  overhead  displays  in  the  attack  center  are  noted  as  trainee 
monitors,  two  blocks  in  the  upper  right  of  Figure  2.  These  are  color 
graphic  19  inch  CRT  monitors  hanging  from  the  overhead. 

f.  Six  time  flag  switches  are  available  in  the  attack  center  and 
moveable  to  various  operator  positions  (upper  right  in  Figure  2).  Trainees 
can  press  the  time  flag  switch  at  any  point  during  an  exercise,  with 
subsequent  recording  of  the  time  at  which  it  was  pushed  for  reference 
purposes  during  exercise  playback. 

g.  The  Classroom  Instructor  Console  (CRC)  (lower  left  of  Figure  2).  It 
is  identical  to  the  IRC,  but  located  in  the  classroom.  The  information 
available  on  the  CRC  is  identical  to  that  available  on  the  IRC,  as  are  the 
controls  and  input  sequences.  The  major  differences  between  the  IRC  and  the 
CRC  center  around  the  capability  on  the  IRC  to  control  the  attack  center 
simulation  exercise  in  real-time.  The  CRC  can  also  control  exercises,  but 
only  during  their  running  in  the  classroom  (e.g.,  during  prebriefings  or 
postbriefings).  The  primary  purpose  of  the  CRC  is  to  control  and  manipulate 
information  on  the  large  screen  classroom  display. 

h.  The  large  screen  classroom  display  is  shown  in  the  lower  left-hand 
corner  of  Figure  2.  It  is  a  large  screen  raster  display  driven  by  an 
Aquastar  projector.  In  addition  to  the  computer  generated  information  that 
can  be  presented,  a  TV  generated  image  of  the  manual  plots,  and  other  status 
displays  from  the  21A41  SCST  MSCG  can  be  presented  on  the  large  screen 
display. 

i. -  Video  camera  (lower  left  in  Figure  2).  A  TV  camera  stand  and  table 
is  located  in  the  classroom.  The  manual  plots  can  be  laid  on  the  table, 
with  their  video  image  picked  up  by  the  camera  and  used  to  generate  the  plot 
image  on  the  large  screen  display.  A  computer  generated  plot  overlay  can  be 
superpositioned  on  the  manual  plot  presentation  of  the  large  screen  display 
for  comparative  purposes. 

j.  A  hard  copy  unit  is  available,  as  part  of  SMARTTS,  in  the  classroom 
to  permit  the  generation  of  a  hard  copy  representation  of  the  large  screen 
display  image. 

The  remaining  hardware  components  in  Figure  2  are  SMARTTS  interface 
equipment  and  existing  attack  center,  sonar,  and  other  21A41  equipment. 

The  locations  of  the  SMARTTS  attack  center  components  are  considered 
critical  to  their  effective  utilization.  Figure  5  shows  the  locations  of 
the  two  overhead  displays  and  the  IRC  within  the  attack  center.  The 
overhead  displays  are  positioned  so  that  at  least  one  display  is  readily 
viewable  by  all  members  of  the  fire  control  party.  Several  potentially 
effective  locations  were  identified  for  the  IRC.  IRC  cable  connections  are, 
therefore,  provided  at  three  different  locations  within  the  attack  center 
facilitating  its  relocation  in  a  short  period  of  time.  The  initial 
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installation  of  the  IRC  is  at  the  location  shown  in  Figure  5,  wnich  is 
believed  to  be  the  best  all  around  choice. 

The  CRC  is  positioned  in  the  classroom  to  one  side  of  the  large  screen 
display.  This  positioning  enables  the  instructor  to  be  in  front  of  the 
students,  and  to  be  able  to  manipulate  the  CRC  while  viewing  both  the 
students  and  the  large  screen  display. 

The  capabilities  of  the  major  components  are  further  addressed  below 
regarding  the  description  of  the  SMARTTS  characteristics. 


3.2  SMARTTS  CHARACTERISTICS 


The  SMARTTS  preprototype  was  designed  to  be  the  training  subsystem  of 
the  21A41  SCST  training  device.  As  such,  it  provides  characteristics  in 
support  of  a  wide  range  of  instructor  functions,  provides  characteristics  to 
assist  the  instructor  in  achieving  an  effective  trainee  interface  within  the 
training  process,  and  provides  rudimentary  characteristics  to  assist  in 
training  system  management.  These  characteristics  support  the  four  major 
functions  of  the  training  process:  (a)  exercise  development,  (b)  monitor  and 
control  of  the  real-time  training  exercise  scenario,  (c)  instructing  the 
student  (e.g.,  briefing),  and  (d)  training  system  management  (Appendix  A 
contains  a  summary  explanation  of  these  functions  and  the  SMARTTS 
characteristics  supporting  each,  taken  from  Hammell,  1982).  Each  major 
SMARTTS  characteristic  is  summarized  below. 


3.2.1  SMARTTS  TRAINING.  SMARTTS  characteristics  support  both  classroom  and 
attack  center  training.  They  can  be  used  simultaneously  in  both  locations, 
supporting  the  four  major  training  process  functions.  Furthermore,  they  are 
intended  to  be  used  to  initiate  the  training  process  prior  to  conducting  the 
real-time  training  exercise  scenario  in  the  attack  center,  to  support  the 
conduct  of  that  exercise  scenario  in  the  attack  center,  and  to  follow-up 
after  the  exercise  scenario  is  completed.  The  variety  of  SMARTTS 
characteristics  to  support  these  vary  considerably,  and  permit  the 
instructor  substantial  flexibility  in  tailoring  the  training  methods  to  the 
various  pertinent  considerations. 


3. 2. 1.1  Exercise  Preview.  Exercise  preview  characteristics  enables  the 
running  of  an  exercise  in  the  classroom  or  in  the  attack  center  (i.e.,  via 
tne  IRC  and  overhead  displays)  without  the  use  of  the  main  simulation/fire 
control  system.  Appropriate  ownship,  target  and  environmental  models  are 
present  in  tne  SMARTTS  software  to  simulate  the  exercise  scenario  without 
tne  use  of  the  main  simulator.  These  preview  characteristics  has  simulation 
models  for  the  following: 

a.  Ownship  actions 

o.  Target  actions 
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c.  Ownship  noise  condition  as  a  function  of  aspect  and  speed 

d.  Target  noise  level  as  a  function  of  specification  and  speed 

e.  Sonar  reception 

f.  Environment 

g.  Target  motion  analysis  computations,  in  terms  of  a  system  solution 
simulation  sensitive  to  ownship  maneuvers,  signal  to  noise  ratio,  target 
maneuvers,  etc. 

h.  The  tactical  variables  and  performance  indicators  are  also 
calculated  during  exercise  preview. 

The  preview  characteristics  are  useful  for  the  instructor  to  develop 
exercises,  in  that  it  enables  the  running  of  each  exercise  at  fast-time  in 
an  off-line  mode.  These  characteristics  are  also  useful  for  previewing 
exercises  shortly  prior  to  running  them  on  the  simulator.  The  greatest  use 
of  the  preview  characteristics  perhaps,  is  during  a  classroom  prebriefing  to 
the  trainees.  Prebriefing  would  logically  precede  the  actual  conduct  of  the 
exercise  in  real-time  on  the  training  device. 

During  the  prebriefing  the  instructor  could  demonstrate  several 
problems,  investigating  acceptable  and  unacceptable  sets  of  tactical 
actions,  or  focusing  on  relevant  Pis  and  tactical  variables.  The  preview 
characteristics  enable  this  type  of  investigative  analysis  to  occur  for  any 
tactical  problem.  They  can  also  be  used  to  investigate  various  problems 
during  the  debriefing  session,  in  a  similar  manner,  prior  to  the  subsequent 
scenario  in  the  attack  center. 


3. 2. 1.2  Exercise  Run.  Many  of  the  SMARTTS  characteristics  have  been 
designed  to  support  the  instructor  during  the  conduct  of  the  real-time 
exercise  in  the  attack  center.  These  capabilities  center  around  the  IRC 
which  presents  the  variety  of  information  needed  by  the  instructor  to 
monitor  and  control  the  exercise,  in  a  location  immediately  adjacent  to  the 

fire  control  party  to  facilitate  his  interaction  with  the  trainees.  The 

input  commands  available  on  the  IRC,  via  the  user-friendly  touch  sensitive 
plasma  panel,  enable  the  instructor  to  control  the  trainer  and  scenario,  and 
to  manipulate  relevant  information  and  displays.  SMARTTS  moves  the 

traditional  information  and  control  capabilities  to  an  effective  and 

convenient  location  for  the  instructor;  generates  and  provides  additional 
necessary  information  to  assist  the  instructor;  and  greatly  extends  the 
instructor's  capabilities  to  control  the  exercise  and  instruct  the  trainee. 

The  two  overhead  displays  in  the  attack  center  can  provide  status 

information  for  viewing  by  the  trainees  during  the  real-time  exercise,  if  so 
desired  by  the  instructor.  The  information  to  be  presented  on  these  j 

displays  is  completely  under  the  instructor's  control  via  the  IRC. 
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3. 2. 1.3  Freeze.  SMARTTS  has  the  capability  to  freeze  time  and  all  SMARTTS 
VAX  and  MSCG  computations  at  any  point  during  a  real-time  exercise  in  the 
attack  center.  Plans  have  been  underway  to  implement  an  automatic  freeze  of 
the  MK 1 3  7  fire  control  system;  this  capability  is  not  currently 
implemented.  Freeze  enables  the  instructor  to  pause  during  a  problem  and  to 
provide  necessary  instruction  before  proceeding  further.  For  example,  after 
ownship  has  been  proceeding  on  a  particular  approach  sequence,  the 
instructor  may  want  to  freeze  the  problem  and  discuss  other  alternative 
actions  (e.g.,  their  pros  and  cons)  and  then  continue  on  with  the  real-time 
scenario.  Another  instance  in  which  the  instructor  may  wish  to  use  freeze 
is  when  the  student  combat  team  has  made  a  fundamental  mistake  early  in  the 
scenario  which  would  inhibit  achieving  the  objectives  the  instructor  is 
focusing  on;  the  instructor  might  freeze  the  problem  to  point  out  the 
mistake  and  get  the  team  back  on  the  right  track  prior  to  proceeding  into 
the  heart  of  the  problem. 

SMARTTS  supports  the  instructional  process  durirtg  the  freeze  by  enabling 
the  generation  of  information  and  its  presentation  on  the  overhead  displays 
in  the  attack  center,  under  the  control  of  the  instructor  via  the  IRC.  The 
overhead  displays  can  provide  information  to  the  team  to  assist  the 
instructor  in  getting  his  points  across. 


3. 2. 1.4  Exercise  Playback.  A  substantial  amount  of  data  is  collected  from 
the  training  device  while  an  exercise  scenario  is  being  run  in  real-time. 
These  data  are  collected  from  the  simulation  computer  (e.g.,  target  sonar 
related  data)  and  from  the  MKl 17  fire  control  system  (e.g.,  system  solution 
data),  as  well  as  other  SMARTTS-generated  data  (e.g.,  subjective 
observations).  The  collected  data  can  be  rerun  during  exercise  playback  in 
real-time,  or  in  accelerated  time  up  to  16x  real-time.  Playback  can  be 
accomplished  in  either  the  attack  center  using  the  overhead  displays 
controlled  by  the  instructor's  remote  console,  or  in  the  classroom  using  the 
large  screen  display  controlled  by  the  classroom  instructor's  console.  All 
the  SMARTTS  display  formats  are  available  for  viewing  during  playback. 

Typically,  the  instructor  would  present  the  problem  on  a  version  of  the 
approach  geoplot  display  format  which  provides  the  geographic  layout  ana 
history  tracks  for  ownship  and  all  other  contacts.  Additional  display 
formats  would  be  used  as  appropriate  with  regard  to  the  particular  training 
oojectives,  problem  scenario,  and  so  on.  The  alternative  tactics 
characteristics  discussed  below  are  also  available  during  playback,  such 
that  the  instructor  can  stop  the  playback  at  any  point  in  time  and 
investigate  alternative  ownship  and  target  actions  from  that  point  on. 


3. 2. 1.5  SMARTTS  Training  Process.  The  playback  characteristics,  supportea 
Dy  the  display  formats  and  other  SMARTTS  features,  are  essential  to  an 
effective  training  process.  Instructional  feedback  is  generally  provideG  in 
the  playback  mode  of  SMARTTS  operation  during  problem  scenario  cebrief,  also 
utilizing  the  other  SMARTTS  capabilities.  The  ideal  training  exercise 
conducted  with  the  21A41A/SMARTTS  would  generally  be  conducted  as  follows: 
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a.  A  prebriefing  session  would  be  conducted  in  the  classroom  using  the 
SMARTTS  preview  characteristics  to  investigate  relevant  tactical  issues  and 
several  problem  scenarios  in  fast-time;  various  display  formats  and  SMARTTS 
characteristics  would  be  drawn  upon  to  dissect  the  scenarios  and  achieve 
technical  understanding  of  the  tactical  issues  involved.  This  would  prepare 
the  trainees  for  the  subsequent  real-time  exercise  scenario  to  be  conducted 
in  the  attack  center. 

b.  A  real-time  exercise  scenario  would  be  conducted  in  the  attack 
center  during  which  the  trainees  would  put  into  practice  the  principles, 
skills,  and  knowledge  they  gained  from  the  prebriefing  session.  This  would 
typically  be  done  at  a  real-time  pace.  The  control  and  monitoring 
characteristics  of  SMARTTS  would  be  used  by  the  instructor,  along  with  other 
characteristics  as  appropriate  (e.g,  overhead  displays  in  the  attack  center 
during  a  freeze). 

c.  A  debriefing  session  in  the  classroom  would  immediately  follow 
completion  of  the  attack  center  exercise  scenario.  The  SMARTTS  playback 
characteristics  would  be  used  to  provide  various  forms  of  feedback  to  the 
trainees.  Typically,  this  would  start  with  a  fast-time  replay  of  the  entire 
exercise  scenario,  using  an  approach  geoplot  display  format  along  with 
several  other  display  formats,  presented  on  the  large  screen  display  in  the 
classroom.  The  instructor  would  then  focus  on  certain  important  segments  of 
the  exercise  to  investigate  aspects  of  trainee  performance  in  greater  detail 
(i.e.,  both  good  and  poor  performance).  The  intention  would  be  to  provide 
specific  feedback  to  the  trainees  regarding  their  performance  and  the 
factors  affecting  it,  and  to  explore  alternative  actions  open  to  them  and 
their  resulting  consequences.  SMARTTS  assists  the  disecting  of  the  exercise 
by  retrieval  of  pertinent  information  that  occurred  throughout  the  exercise 
as  it  was  run  in  real-time  in  the  attack  center. 

d  .  The  debrief  would  move  from  a  focus  on  the  previously  run  attack 
center  exercise  scenario  to  focus  on  subsequent  tactical  issues  of 
interest.  The  subsequent  issues  could  be,  for  example,  offshoots  of  the 
tactical  exercise  to  better  clarify  relationships  between  tactical  variables 
(e.g.,  PCD  and  ownship  speed)  to  dig  deeper  into  particular  aspects  of  the 
training  objectives  currently  being  addressed,  or  to  move  on  to  subsequent 
training  objectives.  This  next  part  of  the  debrief  would  change  into  a 
classroom  discussion,  and  eventually  into  a  prebrief  for  the  next  exercise 
scenario  to  be  run  in  the  attack  center.  The  various  SMARTTS 
characteristics,  such  as  alternative  tactics  and  exercise  preview,  would  be 
used  during  this  phase  of  the  training  process.  Eventually  this  phase  would 
become  step  1  noted  above  in  preparation  for  the  next  exercise  scenario. 

It  should  be  evident  from  the  above,  that  the  SMARTTS  characteristics 
support  both  classroom  and  attack  center  training.  Furthermore,  they  are 
intended  to  be  used  to  initiate  the  training  process  prior  to  conduct  of  the 
real-time  exercise  scenario  in  the  attack  center,  to  support  the  conduct  of 
that  exercise  scenario  in  the  attack  center,  and  to  follow-up  after  the 
exercise  scenario  has  been  completed.  The  variety  of  SMARTTS 
character i sties  to  support  these  vary  considerably,  and  permit  the 
instructor  substantial  flexibility  in  tailoring  the  training  methods  to  the 
various  pertinent  considerations  (e.g.,  training  objectives,  trainee  input 
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characteristics,  etc.)*  The  particular  SMARTTS  characteristics  to  De  used 
will,  of  course,  depend  upon  these  factors. 


3.2.2  PERFORMANCE  INDICATORS.  Performance  indicators  (PI)  are  critically 
important  to  the  effectiveness  of  the  training  process.  The  Pis,  together 
with  other  relevant  tactical  information,  comprise  the  main  body  of 
information  relevant  to  specific  aspects  of  individual  and  team  tactical 
performance.  As  such,  they  constitute  the  trainee  guidance  and  feedback 
information  critical  during  the  training  process.  The  performance 
indicators  are  generated  by  SMARTTS  based  on  information  collected  from  the 
MK117  fire  control  system,  the  main  simulation  computer,  and  the 
instructor.  Each  performance  indicator  addresses  a  specific  tactical  aspect 
of  the  problem  relevant  to  various  training  objectives.  The  Pis  have  oeen 
developed  in  two  groups:  (a)  objective  Pis  which  are  automatically  generated 
by  the  computer,  and  ( b)  subjective  Pis  which  are  manually  entered  into  the 
computer  by  the  instructor,  based  on  his  observations.  The  algorithms  used 
to  calculate  the  objective  Pis  use  tactical  data  and  other  parameters  as 
inputs  (e.g.,  rate  of  change  of  bearing,  time  and  distance  to  the  target, 
target  angle  on  the  bow,  target  sonar  performance,  etc.)  which  are  normally 
generated  throughout  any  trainer  exercise.  Many  of  the  objective  Pis  are 
calculated  once  per  minute  and  recorded,  enabling  their  display  at  various 
times  during  or  following  the  exercise  in  graphical  format.  All  objective 
PI  values  are  available  for  presentation  at  or  near  the  time  of  occurrence 
of  their  measured  events  during  the  tactical  exercise,  when  selected  by  the 
instructor  (i.e.,  presented  on  the  instructor  consoles  or  overhead  and  large 
screen  displays).  Time  history  plot  displays  of  Pis  are  also  available, 
showing  their  instantaneous  value  as  well  as  a  plot  of  their  previous  values 
during  the  scenario  (see  information  displays  below).  The  objective  Pis  are: 

a.  Probability  of  counterdetection 

b.  TMA  system  solution  -  course  accuracy 

c.  TMA  system  solution  -  range  accuracy 

d.  TMA  system  solution  -  speed  accuracy 

e.  TMA  system  solution  -  depth  accuracy 

f.  Approach  position  -  track  angle 

g.  Closing  rate  -  across  line  of  sight 

h.  Closing  rate  -  in  line  of  sight 

i.  Time  to  station  -  (i.e.,  time  to  approach  position) 

j.  Torpedo  run 

k.  Periscope  exposure  time 

l.  Kalman  automatic  sequential  target  motion  analysis  (KAST)  reset  times 
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The  subjective  Pis  involve  instructor  observations  of  particular  factors 
that  occur  during  the  exercise,  such  as  the  trainee  team's  minimum/maximum 
values  for  the  initial  target  motion  analysis  (TMA)  range  estimate,  or  the 
overall  effectiveness  of  fire  control  team  communication.  These  are 

relevant  Pis  that  cannot  be  readily  calculated  by  automated  means,  but  are 
of  vital  importance  to  the  training  objectives.  The  subjective  PI  values 

are  determined  by  the  instructor  and  entered  into  the  SMARTTS  computer  at 
the  IRC,  and  subsequently  stored  for  later  retrieval.  Occasionally,  a  cue 
or  an  alert  is  provided  to  the  instructor  via  the  IRC  to  indicate  a 
particular  occurrence  (e.g.,  PCD  exceeding  a  preset  value)  after  which  the 
instructor  may  wish  to  observe  certain  aspects  of  performance  and  enter  a 
subjective  PI.  The  instructor's  command  indicating  his  desire  to  enter  a 
subjective  PI  causes  automatic  selection  of  the  appropriate  data  entry 
tableau  on  the  plasma  entry  device.  This  facilitates  instructor  entry  and 
recording  of  subjective  PI  values.  The  subjective  Pis  can  be  displayed  for 
feedback  purposes  in  a  manner  similar  to  that  for  the  objective  Pis, 

although  they  usually  represent  discrete  points  when  looked  at  over  time, 

rather  than  a  smooth  history  track.  The  set  of  subjective  Pis  provided  on 
the  preprototype  SMARTTS  are: 

a.  TMA  initial  range  estimate,  minimum  and  maximum. 

b.  Recognition  of  target  maneuvers,  including  time  to  detect  the 
maneuver  and  the  accuracy  of  evaluation. 

c.  Periscope  angle  on  the  bow  recognition. 

d.  Periscope  safety  sweep  procedure. 

e.  Periscope  search  technique. 

f.  Plot  coordinator  effectiveness. 

g.  Fire  control  coordinator  effectiveness. 

h.  Fire  control  party  interaction,  including  intraparty  communication 
and  interparty  communication  (i.e.,  between  fire  control  party  and  other 
teams  such  as  the  sonar  party). 

Tne  flexibility  of  the  SMARTTS  preprototype  software  design  to  enable 
evolution  of  system  characteristics  was  given  high  priority.  It  is  the 
intention  that  the  specific  characteri sties  of  SMARTTS  (e.g.,  performance 
indicators,  display  formats)  can  be  easily  modified  to  keep  pace  witn  the 
changing  training  objectives,  tactical  doctrine,  training  methods,  and  so 
on.  Hence,  the  above  list  of  objective  and  subjective  Pis  is  not  intended 
to  represent  the  comprehensive  list  that  should  be  used  in  submarine  tactics 
training.  Ratner,  it  represents  the  first-cut  at  a  set  of  acceptable  Pis. 
It  is  fully  expected  that  these  Pis  will  be  improved  as  a  result  of  tneir 
use  in  training,  and  that  additional  Pis  will  be  developed  and  implemented 
into  the  SMARTTS  preprototype. 
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3.2.3  TACTICAL  VARIABLES.  The  Pis  summarized  aDove  represent  specific 
indicators  of  tactical  performance  for  individuals  and  for  the  team.  They 
are  typically  used  to  provide  feedback  to  the  trainees  immediately  following 
a  scenario,  as  well  as  to  investigate  alternative  tactical  actions  prior  to 
and  following  a  scenario,  contrasting  the  results  in  performance  trade-offs 
between  the  various  Pis.  Several  tactical  variables  ( i . e . ,  parameters 
normally  occurring  in  the  tactical  encounter)  are  also  critically  important 
information  sources  to  be  used  during  preproblem  and  postproblem  briefings. 
The  tactical  variables  are  based  on  the  observed  values  of  specific 
parameters  as  they  occur  during  the  exercise.  The  tactical  variables  are 
recorded  in  a  manner  similar  to  that  of  the  objective  Pis,  and  likewise 
available  for  display.  That  is,  all  tactical  variables  are  available  for 
presentation  on  the  displays  during  tactical  exercises  under  instructor 
control,  and  in  accordance  with  the  TAT  characteri sties  involved.  Tney  can 
be  presented  on  the  IRC  and  the  CRC,  as  well  as  the  overhead  displays  in  the 
attack  center  and  the  large  screen  display  in  the  classroom.  The  tactical 
variaDles  recorded  by  the  preprototype  SMARTTS  are: 

a.  Ownship  course 

o.  Cwnship  speed 

c.  Ownship  depth 

d.  Target  course 

e.  Target  speed 

f.  Target  depth 

g.  Target  lead/lag  angle' 

•  h.  Target  angle  on  the  bow 

i.  Target  true  bearing 

j.  Target  bearing  rate 


< .  Target  range 
1 .  Target  range  rate 

in.  Target  signal  to  noise  ratio  for  assigned  trackers 
n.  Target  speed  across  the  1 ine-of-sight 


3.2.4  I  INFORMATION  DISPLAYS.  Much  of  the  work  performed  by  SMARTTS  is  tne 
collection  of  data  from  various  sources,  transformation  of  tnat  oata  in:,, 
meaningful  tactical  variables,  performance  indicators,  and  other  forms,  anc 
presentation  of  relevant  information  on  displays  to  the  instructor  anG 
trainees.  The  various  forms  of  information  presented  to  the  instructor 
directly  support  nis  functions  of  exercise  development,  monitoring  ar.c 
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control  of  training  on  the  device,  and  control  of  information  presentation 
to  the  trainees.  The  information  presented  to  the  instructor  in  addition  to 
the  performance  indicators  and  tactical  variables,  consists  of  the  ownship, 
target,  and  environmental  parameters  essential  to  exercise  setup  and 
control.  The  information  presented  to  the  trainees,  on  either  the  overhead 
displays  in  the  attack  center  or  the  large  screen  display  in  the  classroom, 
is  primarily  used  to  disect  and  investigate  particular  aspects  of  tactical 
problems,  to  provide  status  information  concerning  the  particular  exercise 
and  scenario  run  on  the  training  device,  and  to  provide  feedback  information 
concerning  the  myriad  of  details  regarding  tactical  performance  during  the 
scenario.  The  information  presented  to  the  trainees  generally  consists  of 
the  performance  indicators  and  tactical  variables.  The  SMARTTS  information 
display  characteristics,  for  both  the  instructor  and  trainees,  are  intended 
for  use  during  prebriefing  sessions  prior  to  conduct  of  the  exercise 
scenario  in  the  attack  center;  during  the  conduct  of  the  exercise  scenario 
in  the  attack  center;  and  immediately  following  completion  of  the  exercise 
scenario.  In  addition,  exercise  development  display  characteristics  are 
available  to  the  instructor  in  an  off-line  context.  It  should  be  noted  that 
a  variety  of  processing  capabilities  has  been  incorporated  into  SMARTTS  to 
manipulate  information  for  display.  These  characteristics  include,  for 
example,  ownship,  target,  and  environmental  models  to  support  classroom 
investigations.  Several  of  the  major  processing  characteristics  are 
summarized  later  in  this  section. 

The  information  display  characteristics  are  summarized  below  in  terms  of 
(a)  the  display  formats,  and  (b)  their  intended  use  during  the  training 
process.  The  majority  of  display  formats  available  on  SMARTTS  are  designed 
to  serve  both  the  instructor  and  the  trainees,  during  the  various  phases  of 
the  training  process  (e.g.,  classroom  briefing,  exercise  scenario  monitoring 
in  the  attack  center).  The  information  display  formats  are  designed  for 
both  general  and  specific  purposes,  with  the  former  useful  in  a  variety  of 
training-related  applications,  and  the  latter  tailored  to  specific  training 
objectives  and/or  tactical  exercises. 

The  available  SMARTTS  display  formats  have  two  general  purposes.  One  is 
to  provide  status  information  regarding  various  aspects  of  the  tactical 
problem  (e.g.,  geographic  picture  of  ownship,  target,  and  other  vessels  in 
the  area;  system  solution  accuracy  at  a  particular  point  in  time).  The 
other  is  to  provide  performance-related  feedback  to  the  trainees  regarding 
their  actual  actions  during  a  particular  scenario,  or  regarding  alternative 
sets  of  actions  available  in  various  tactical  scenarios.  Often,  the  same 
display  formats  are  used  for  both  of  these  purposes.  Hence,  the  summary 
provided  herein  does  not  distinguish  the  purpose  for  each  display  format. 
Furthermore,  some  display  formats  are  intended  for  information  to  be 
provided  to  individual  trainees,  while  others  are  intended  for  information 
to  be  provided  to  the  complete  team,  while  still  others  provide  information 
for  the  instructor  and  instructional  staff.  Finally,  many  of  the  display 
formats  are  modularly  constructed  such  that  a  particular  format  may  be 
composed  of  several  blocks,  for  each  of  which  several  alternatives  are 
available.  Hence,  for  example,  geographical  plot  can  be  presented  along  up 
with  associated  performance  indicator  summaries;  alternatively,  that  same 
geographic  plot  can  be  displayed  along  with  a  1  ine-of-sight  diagram  from 
ownship  to  the  target.  Various  combinations  of  the  display  blocks  are 


NAVTRAEQUIPCEN  80-C-0079 


available  to  be  presented  on  a  particular  display  format  at  any  point  '.n 
time. 

Tne  summary  presented  below  is  not  intendea  to  be  comprenensi ve  witn 
regard  to  all  of  tne  display  formats  and  combinations  availaDle  on  SMAkTTS. 
Rather,  it  presents  a  representative  overview  of  tnose  that  are  available. 

a.  Geoplot  Display  --  This  format  (Figure  6)  provides  a  geographical 
plot  of  ownship,  the  target  and  other  contacts,  weapons,  ana  their  nistory 
tracks.  Several  characteristics  options  are  availaDle  for  configuring  tnis 
particular  display  format,  such  as  the  range  scale  for  the  geographic  area 
(e.g.,  a  32  nautical  mile  scale  is  in  the  Figure  6  example).  Immediately  to 
the  right  of  the  geograpnical  plot  is  an  area  available  for  display  of 
associated  information.  In  the  Figure  6  example,  six  tactical  varieties 
presenting  current  status  information  are  being  displayed.  This  area  could 
also  be  used  for  displaying  up  to  three  sets  of  performance  indicators 
(selectable  by  the  instructor).  Figure  7  shows  a  geographical  plot  witn  an 
associated  1 ine-of-sight  diagram,  showing  the  relationship  between  ownsni? 
and  the  target.  The  1  ine-of-sight  diagram  is  a  plot  often  constructea  ar.o 
used  by  the  submarine  fire  control  party  during  a  tactical  engagement.  Its 
use  associated  with  the  geographical  plot  on  this  particular  display  format 
is  to  provide  status  information  to  the  instructor  during  a  particular 
exercise  on  the  training  device.  Additionally,  this  display  formas 
(approach  geoplot  and  1 ine-of-sight)  could  also  be  used  to  provide  feecbacx 
to  the  trainees  during  a  freeze  in  the  exercise,  or  during  a  postproblem 
feedback  session  in  the  classroom.  When  used  to  monitor  the  status  of  an 
ongoing  problem,  this  display  updates  all  parameters  in  real-time.  When 
used  to  provide  postproblem  feedback,  the  scenario  can  be  replayed  at  rates 
ranging  from  real-time  to  16x  real-time,  with  the  display  upaating 
accordingly.  This  display  format  is  also  extremely  useful  during  exercise- 
development  when  the  instructor  is  concerned  with  the  interactions  between 
the  various  platforms,  tactical  variables  generatea,  and  so  on.  Please  note 
that  a  line  of  alphanumeric  information  can  be  presented  at  the  very  bottom 
of  the  display  in  Figures  6  and  7.  This  area  is  reserved  for  presenting 
cues  to  the  instructor  during  the  ongoing  problem  (e.g.,  alerts  regarding 
target  actions,  performance  indicators  exceeaing  a  preset  thresnold  level, 
and  so  on) . 

o.  Target  Status  —  This  display  format  (Figure  8)  provides  several 
types  of  information  relating  to  target  status.  The  block  of  information  on 
tne  left  side  of  the  display  presents  the  target  status  in  terms  of  system 
versus  actual  target  motion  analysis  solution  information  for  seven  relevant 
tactical  variables.  This  information  allows  the  instructor  to  view  c 
summary  of  target  tactical  actions,  and  enaoles  him  to  monitor  the 
difference  between  the  actual  target  parameters  ana  tne  solution  acnievea  ;>y 
the  fire  control  party  during  the  scenario.  Several  different  blocxs  of 
information  are  available  to  present  on  tne  right  side  of  tnis  display 
format.  The  two  blocks  in  Figure  8  address  tne  performance  inaicator 
probability  of  counterdetection,  the  received  sonar  signal  to  noise  ratio, 
and  several  ownship  parameters  (e.g.,  ownship  course).  The  lower  part  or 
tnis  status  display,  which  is  blank  in  Figure  8,  could  present  information 
relevant  to  actions  by  the  automatic  interactive  target  (AIT)  when  an  AIT  is 
designated  ana  turned  on  (refer  to  Section  3.2.9  for  a  description  of  tne 
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All).  Tins  display  format  would  typically  be  used  during  the  conduct  of  ; 
scenario  on  the  training  device  to  monitor  the  system  solution  status  as¬ 
certain  other  tactical  variables.  Of  course,  it  could  also  be  used  for 

off-line  exercise  construction,  classroom  monitoring  and  classroom  briefing. 

c.  Time  Function  --  This  display  (Figure  9)  plots  a  selected  PI  or 

tactical  variable  on  the  vertical  axis,  versus  problem  time  on  tr.e 
horizontal  axis.  A  combined  total  of  up  to  three  Pis  and  tactical  variables 
can  be  displayed  simultaneously  on  this  display  format.  Each  of  the  three 
curves  is  presented  in  a  different  color.  The  time  function  display  format 
is  used  primarily  to  provide  feedback  information  to  the  trainees  subsequent 
to  completing  an  exercise  scenario  in  the  attack  center.  This  format  is 
also  very  useful  during  preexercise  briefings.  A  vertical  cursor,  which  is 
positioned  near  time  0120  in  Figure  9,  can  be  manually  positioned  to  any 
time  in  the  exercise,  or  allowed  to  automatically  follow  the  current 

exercise  time.  The  value  of  each  of  the  three  displayed  parameters  at  tr,e 
time  indicated  by  the  vertical  cursor  is  presented  on  tne  display  in  a 
digital  readout  form  to  the  left  of  the  vertical  axis.  The  time  cursor  is 

placed  at  0114  minutes  in  the  example  of  Figure  9,  with  the  selected 

parameter  (TSP)  value  reading  999.  If  this  cursor  were  moved  to  different 
times  the  corresponding  values  would  be  readout. 

d.  Trigraph  --  The  trigraph  display  places  three  time  function  plots  on 

a  single  display  surface  (Figure  10).  One  to  three  Pis  or  tactical 

variables  can  ue  displayed  on  each  of  the  three  plots  on  the  trigraph. 

Figure  10  shows  a  single  parameter  graphed  on  each  of  the  plots.  Tne 
vertical  cursor,  which  can  be  manually  or  automatically  positioned,  is 

provided  on  the  trigraph  in  a  manner  similar  to  that  for  the  time  function 
display  (see  above  description).  As  shown  in  the  Figure  10  example,  the 
vertical  cursor  is  positioned  at  time  0009,  with  three  corresponding  values 
displayed  for  the  three  parameters  plotted.  It  should  be  noted,  that  a 
group  of  up  to  three  Pis  and  tactical  variables  can  be  allocated  to  each  of 
three  data  blocks  for  a  total  of  nine  parameters  simultaneously  displayed. 
When  selecting  parameters  for  certain  displays  such  as  the  trigrapn,  tne 
instructor  can  select  the  specific  Pis  or  tactical  variables  he  wishes  to 

present,  or  he  could  select  a  particular  data  blocx  or  blocks  he  wishes  to 

present.  In  the  latter  case,  the  Pis  previously  designated  as  comprising 
the  three  available  data  blocks  would  be  automatically  presented  on  the 
display  (i.e.,  three  Pis  are  automatically  designated  in  each  of  three 

blocks  as  part  of  each  exercise,  or  manually  designated  at  any  time  by  the 
instructor),  lienee,  for  example,  if  three  parameters  were  assigned  to  eac: 
data  block,  and  the  instructor  selected  three  data  blocks  for  presentation 
on  the  triyrapn,  nine  parameters  would  be  simultaneously  displayed.  Inis 
feature  enables  the  instructor  to  rapidly  select  the  often  aseu  •’  1  s  arc 
tactical  variables  as  groups  without  having  to  manually  request  eacn 
parameter . 

e.  Parametric  Plot  --  The  parametric  plot  display  presents  a  two  -xis 

brapnical  presentation  of  the  relative  values  of  two  time  a)i,ti;.uu.-s  PI.,  ..." 
tactical  variables  on  the  same  graph.  One  parameter  is  plotted  on  me  - 

axis  while  tne  second  parameter  is  plotted  on  the  X  axis.  Tne  plot  - 
similar  to  mat  of  the  time  function  graph,  witn  tne  exception  that  m- 
horizontal  axis  parameter  is  selectable  ratner  tnan  fixed  as  preo iem  time. 
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The  above  display  format  examples  are  representative  of  those  currently 
available  in  SMARTTS.  SMARTTS  contains  additional  display  formats,  as  well 
as  a  variety  of  characteristics  associated  with  all  displays.  As  noted 
earlier,  most  of  the  display  formats  available  under  SMARTTS  have  multiple 
uses,  including  uses  by  the  instructor  for  monitoring  and  exercise 
development,  and  for  briefing  and  feedback  information  to  the  trainees. 
Usually,  the  specific  display  characteristics  (e.g..  Pi's)  are  tailored  to 
the  particular  purposes  of  the  training  exercise.  The  trigraph  display,  for 
example,  is  often  used  as  a  means  of  providing  detailed  feedback  to  the 
trainees  following  completion  of  an  exercise/scenario  on  the  training 
device.  When  used  for  this  purpose  relevant  Pis  and  tactical  variables  are 
displayed  as  a  function  of  time  to  illustrate  (a)  details  of  the  tactical 
problem  as  it  developed,  and  (b)  relationships  between  the  different 
tactical  variables  and  Pis  (e.g.,  relationship  between  probability  of 
counterdetection  and  ownship  speed).  The  trainees  are  able  to  see  how  a 
change  in  ownship  speed  affected  a  change  in  probability  of 
counterdetection,  for  example,  since  both  occurred  at  approximately  the  same 
time  with  the  former  being  a  function  of  the  latter.  The  trigraph  display 
is  also  useful  by  the  instructor  to  monitor  aspects  of  the  problem  as  it  is 
developing  in  real-time  in  the  attack  center.  When  the  trigraph  is  used  in 
this  manner,  the  vertical  time  cursor  is  usually  locked  to  the  current  time 
and  updates  the  digital  readouts  in  real-time  as  the  problem  progresses. 
The  cursor,  therefore,  provides  the  instructor  with  a  digital  readout  of  the 
current  value  of  up  to  nine  different  performance  indicators  and  tactical 
variables  at  once,  and  also  allows  him  to  view  a  historical  plot  of  their 
value  from  the  onset  of  the  exercise,  or  from  any  particular  point  in  the 
exercise.  These  plots  enable  the  instructor  to  observe  trends  in 
performance,  such  as  a  steadily  increasing  probability  of  counterdetection 
over  time.  Both  the  instantaneous  and  time  history  data  are  valuable  to  the 
instructor  to  interpret  the  evolving  tactical  situation  and  the  trainees' 
activities,  and  hence  to  direct  the  problem  or  provide  feedback  (i.e., 
either  immediate  or  postproblem  delayed)  regarding  specific  aspects  of  the 
problem  and  trainee  performance.  Several  additional  display  formats  are 
discussed  later  with  regard  to  particular  SMARTTS  characteristics. 

The  SMARTTS  software  component  that  implements  the  displays  was  flexibly 
designed  to  permit  modification  of  the  display  formats.  A  major  intention 
of  the  SMARTTS  design  is  to  enable  the  system  to  evolve  as  the  tactical  and 
training  needs  and  methods  change.  Hence,  display  formats  are  intended  to 
be  developed  and  implemented  with  regard  to  particular  training  objectives; 
the  software  is  designed  to  encourage  this  type  of  modification. 


3.2.5  ALTERNATIVE  TACTICS.  Learning  by  example  has  been  tradi tional ly 
accepted  as  an  effective  training  methodology.  Witness  the  extensive  use  of 
examples  during  lectures  and  homework  iroblems.  The  alternative  tactics 
feature  of  SMARTTS  provides  a  computer-based  capability  to  rapidly  generate 
and  investigate  example  problems.  Prior  to  a  real-time  exercise  on  the 
training  device  the  instructor  may  wish  to  explore  particular  concepts  in 
the  classroom  (e.g.,  various  barrier  patrol  patterns  to  intercept  a 
transiting  submarine)  by  illustrating  the  impact  of  ownship  or  target 
alternative  actions  on  the  performance  indicators  and  tactical  variables  of 
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interest.  This  would  provide  the  trainees  with  an  understanding  of  the 
trade-offs  associated  with  the  various  alternative  actions  available  to  them. 

SMARTTS  can  generate  several  alternative  sets  of  actions  for  a 
particular  problem  (i.e.,  both  ownship  and  target  alternative  actions), 
generate  the  associated  performance  indicators  and  tactical  variables, 
present  the  relevant  information  summaries  on  several  display  formats,  and 
enable  the  instructor  to  draw  comparisons  between  the  various  alternatives 
with  regard  to  the  objective  PI  and  tactical  variable  developments  and 
outcomes.  Similarly,  during  a  postscenario  feedback  session  in  the 
classroom  the  instructor  may  wish  to  illustrate  alternative  sets  of  actions 
for  the  particular  problem  encountered  during  the  real-time  scenario  in  the 
attack  center.  For  example,  after  a  2-hour  exercise  during  which  ownship 
was  on  barrier  patrol  and  encountered  a  transiting  enemy  submarine,  took 
appropriate  action  to  approach  and  attack,  and  eventually  completed  the 
scenario,  the  instructor  may  wish  to  illustrate  the  effectiveness  of  other 
sets  of  tactical  actions  for  that  particular  problem.  These  alternative, 
sets  of  actions  may  be  equally  effective,  less  effective,  or  more 
effective.  Also,  it  is  often  very  difficult,  if  not  impossible,  to 
determine  the  "best"  set  of  tactical  actions  since  many  interacting  and 
often  conflicting  changing  the  factors  are  involved.  From  a  training 

standpoint  it  is  often  more  important  that  the  trainees  understand  the 
various  trade-offs  involved  with  the  particular  actions  they  took  in 
comparison  with  other  available  actions,  rather  than  to  be  told  that  a 

particular  set  is  the  best.  To  adequately  explore  these  trade-offs  and 
alternatives,  the  use  of  examples  providing  objective  PI  and  tactical 
variable  information  regarding  each  in  a  comparative  format  is  extremely 
useful.  The  knowledge  gained  from  this  type  of  activity  constitutes 

learning. 

The  alternative  tactics  characteristics  is  also  useful  by  the  instructor 
during  the  exercise  development  process.  During  the  design  and  validation 
of  exercise  scenarios  it  is  often  necessary  for  the  instructor  to  explore 
alternative  sets  of  target  and  ownship  actions  to  arrive  at  the  final 

design.  The  alternative  tactics  characteristics  provide  a  convenient  means 
of  setting  up  and  evaluating  these  alternatives. 

The  alternative  tactics  characteristics  include: 

a.  Up  to  five  future  ownship  legs  can  be  projected  (i.e.,  each  leg 
constitutes  a  segment  of  a  scenario  when  ownship  is  on  a  constant  course  and 
at  a  constant  speed). 

b.  Up  to  three  target  legs  can  be  projected.  The  computer 
automatically  projects  ownship  and  target  tracks  in  fast-time  in  accoraance 
with  the  entered  information. 

c.  Several  tactical  variables  are  calculated  through  the  projected 
target  and  ownship  maneuvers  (e.g.,  bearing,  bearing  rate,  range). 

d.  Several  objective  performance  indicators  are  calculated  through  tne 
target  and  ownship  maneuver  legs  (e.g.,  target  motion  analysis  solution 
accuracy,  sucn  as  range;  probability  of  counteraetection) . 
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e.  The  targets  for  which  no  alternative  actions  are  entered  are 
automatically  projected  at  their  current  course  and  speed  through  the  time 
of  the  complete  alternative  tactics  projection. 

f.  Projection  times  can  range  from  10  to  120  minutes. 

Several  display  formats,  similar  to  those  presented  earlier,  have  been 
specifically  designed  to  present  the  comparative  alternative  tactics 
information.  Figure  11  shows  an  approach  geoplot  display  format  for 
alternative  tactics.  The  geographical  plot  area,  in  this  particular 
example,  is  presenting  three  different  alternatives  for  ownship  as  indicated 
by  the  three  track  projections  in  the  figure  emanating  from  the  ownship 
position.  No  alternative  actions  for  the  target  are  projected  in  this 
particular  example,  with  the  resultant  target  track  projection  based  on  the 
target's  current  speed  and  course.  The  right  side  of  the  display  format 
presents  six  PI  and  tactical  variable  parameters  for  each  of  the  three 
alternative  options  (i.e..  Option  A,  B,  and  C),  enabling  their  comparison 
for  each  alternative  set.  The  Pis  and  tactical  variables  displayed  in  this 
data  area  are  selectable.  In  the  particular  example  presented  in  Figure  11, 
the  data  in  the  blocks  represent  the  values  at  the  ends  of  Legs  3,  5,  and  5 
for  the  three  alternative  sets  (i.e.,  options  A,  B,  and  C),  respectively. 
For  example,  range  to  the  target  at  the  end  of  Leg  3  under  Option  A  would  be 
6,344  yards;  range  to  the  target  at  the  end  of  Leg  5  under  Option  B  would  be 
24,114  yards;  range  to  the  target  at  the  end  of  Leg  5  under  Option  C  wcula 
be  16,800  yards.  Coordination  between  the  geoplot  tracks  and  the  data 
blocks  is  also  color  coded  to  facilitate  analysis. 


3.2.6  INSTRUCTOR  CUES.  Cue  information  is  provided  to  the  instructor  in 
real-time  via  the  instructor  remote  console  in  the  attack  center  during  an 
exercise  scenario.  The  alphanumeric  cue  information  is  provided  in  a  single 
line  at  the  bottom  of  the  CRT  display.  Several  types  of  cues  are  provided 
by  SMARTTS: 

a.  Notes  --  Notes  are  cues  that  have  been  embedded  in  the  script  of 
each  exercise  scenario  prior  to  the  start,  such  that  they  are  displayed  at 
predetermined  times.  An  example  of  a  note  is  the  listing  of  trainee 
objectives  that  is  displayed  after  initiation  of  the  exercise. 

b.  Alert  —  An  alert  is  a  cue  provided  to  the  instructor  on  the  basis 
of  one  or  more  events  occurring  during  the  scenario.  The  alert  times  are 
not  predetermined,  but  rather  depend  upon  the  occurrence  of  particular 
events.  For  example,  an  alert  may  be  given  when  the  probability  of 
counterdetection  exceeds  a  predetermined  level  (e.g.,  50  percent). 
Likewise,  an  alert  would  be  given  when  the  automatic  interactive  target 
initiates  a  maneuver.  The  instructor  can  preselect  the  threshold  value  that 
triggers  the  alert. 

c.  Alarms  --  Alarm  cues  are  given  following  a  SMARTTS  fault.  An 
example  is  when  a  particular  display  device  is  out  of  order,  or  when  the 
SMARTTS-to-MSCG  linkup  can  not  be  accomplished. 
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The  cues  are  displayed  as  they  occur,  and  in  a  priority  order  (e.g., 
alarms  have  the  highest  priority,  with  notes  having  the  lowest  priority). 
When  multiple  cues  are  received  they  are  stored  in  order  in  a  file,  with  the 
highest  priority  and  earliest  cue  presented  on  the  screen.  After  the 
instructor  acknowledges  a  particular  cue,  it  is  removed  from  the  screen,  and 
the  next  cue  in  line  (i.e.,  if  additional  cues  exist)  becomes  displayed. 
Cue  information  assists  the  instructor  in  several  ways:  first,  by  storing 
predetermined  notes  for  him  to  be  brought  up  at  appropriate  times  during  the 
exercise  scenario;  and  second,  by  monitoring  critical  events  during  the 
exercise  scenario,  and  thus  reducing  the  instructor's  monitoring  load. 


3.2.7  EXERCISE  LIBRARY.  A  set  of  exercises  to  support  a  set  of  tactical 
training  objectives  has  been  developed  under  SMARTTS.  These  exercises, 
including  all  relevant  parameters,  are  stored  on  the  computer.  To  initiate 
an  exercise,  the  instructor  can  select  the  desired  exercise  via  the  IRC  and 
have  it  automatically  loaded  and  initiated  on  the  trainer.  This  capability 
greatly  reduces  the  amount  of  time  the  instructors  have  typically  spent 
manually  loading  the  various  parameters  necessary  to  setup  an  exercise 
scenario. 

Many  aspects  of  the  exercise  scenario  can  be  previewed  by  the  instructor 
prior  to  loading  on  the  trainer.  The  instructor  has  the  capability  to 
modify  significant  tactical  parameters  immediately  prior  to  their  automatic 
loading.  If  he  chooses  to  modify  scenario  parameters,  he  can  then  save  the 
exercise  scenario  if  so  desired,  and  add  it  to  the  exercise  library.  In 
this  manner,  as  new  exercises  are  manually  developed  either  off-line  or 
immediately  prior  to  initiating  an  exercise  scenario  for  a  particular  group 
of  trainees,  they  can  be  readily  added  to  the  exercise  library  for  later 
selection  and  automatic  loading. 


3.2.8  TIME  FLAG  UNITS.  Six  time  flag  units  are  available  in  the  attack 
center,  five  of  which  are  allocated  to  trainee  operator  stations  and  one 
allocated  to  the  instructor's  console.  The  time  flag  unit  is  a  small  button 
which  can  be  pushed  by  trainees  and  the  instructor  at  any  time  during  an 
exercise  scenario.  The  time  at  which  each  time  flag  unit  is  pushed  is 
automatically  recorded  in  the  SMARTTS  data  base.  These  times  can  later  be 
retrieved  during  playback  in  various  ways.  They  serve  to  indicate  a  point 
in  the  exercise  scenario  at  which  the  instructor  and/or  trainees  wish  to 
discuss  some  issue.  In  essence,  they  serve  as  a  cue  during  playback. 


3.2.9  AUTOMATIC  INTERACTIVE  TARGET.  The  vast  majority  of  SMARTTS 
capabilities  are  aimed  at  supporting  the  training-rel ated  aspects  of  the 
training  device  and  training  system,  as  discussed  above.  SMARTTS  generally 
does  not  provide  capabilities  to  improve  the  simulation  aspects  of  the 
training  device.  The  automatic  interactive  target  (AIT)  is  an  exception,  in 
that  its  characteristics  serve  a  dual  purpose:  (a)  to  reduce  the 
instructor's  work  load  (i.e.,  analyses  have  shown  that  the  instructor  spenas 
a  considerable  amount  of  time  in  controlling  the  targets),  and  (o)  to 
improve  the  simulation  quality  of  enemy  targets.  The  automatic  interactive 
target  is  a  computer  controlled  target  model.  This  model  is  composed  of  a 
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decision  structure  for  an  enemy  submarine  platform.  The  AIT  characteristics 
were  developed  from  the  best  intelligence  information  availaole  to  the 
SMARTTS  project  concerning  the  particular  submarine  platform  implemented. 
The  AIT  acts  as  an  intelligent  adversary,  whose  ship  has  capabilities 
equivalent  to  those  of  the  actual  at-sea  enemy  submarine  platform.  The  AIT 
reacts  to  ownship's  actions,  as  he  would  be  expected  to  receive  information 
through  the  environment  and  intelligence  sources,  interpret  that 
information,  and  select  his  tactical  actions.  The  manual  method  of 
controlling  target  actions  by  the  instructor,  may  have  undesirable  effects, 
such  as  instructor  bias  in  his  perception  of  target  actions  and  their 
probability  of  occurrence;  it  also  creates  some  competition  between  the 
instructor/target  and  the  ownship  team  during  the  exercise  scenario. 
Although  manual  control  of  the  target  by  the  instructor  may  be  necessary  in 
certain  operations,  the  AIT  can  often  alleviate  instructor  control  and 
present  a  probabilistically  realistic  target. 

The  AIT  structure  is  based  on  a  dynamic  probabilistic  decision  model.  A 
high  level  overview  of  that  model  is  presented  in  Figure  12.  This  diagram 
shows  major  decision  points  that  occur  throughout  the  exercise  scenario,  for 
which  alternative  sets  of  target  actions  are  associated.  The  AIT  has  three 
levels  of  competency,  ranging  from  basically  capable  to  highly  capable.  The 
actions  available  to  the  AIT,  and  their  probabilities  of  occurrence,  are 
keyed  to  the  AIT's  level  of  competence  as  well  as  other  factors  (e.g., 
operational  mission).  The  typical  mission  evolution  of  the  AIT  would 
initiate  at  the  top  of  the  block  diagram  beginning  with  the  block  (see 
Figure  12)  tactical  parameters  and  follow  through  based  on  events  that 
occurred  during  the  scenario,  the  information  received  and  its 
interpretation  on  a  probabilistic  basis  by  the  AIT,  and  the  action  selected 
on  a  probabilistic  basis.  The  typical  exercise  would  likely  follow  through 
to  either  evasion  or  attack. 

Information  regarding  AIT  sensor  monitoring  and  tactical  activities  is 
provided  to  the  instructor  for  overseeing  the  target  actions.  Alerts  are 
provided  on  the  instructor's  console  in  the  attack  center  informing  him  of 
decisions  being  made  by  the  AIT,  and  impending  AIT  actions.  The  instructor 
can  override  planned  actions  of  the  AIT  prior  to  their  occurrence.  The 
instructor  can  also  initiate  preset  actions  by  the  AIT,  such  as  starting  a 
particular  baffle  clearing  sequence.  In  this  manner,  the  instructor  is  not 
overburdened  by  evaluating  in  detail  the  actions  of  ownship,  their  impact  on 
the  target,  and  the  actions  the  target  should  take,  but  rather  has  his 
target  control  work  load  reduced  to  primarily  a  monitoring  function,  with 
override  action  if  necessary. 

The  AIT  perfoi  mance  profile  is  controlled  by  a  role  situation  and 
mission  assignment.  This  initializes  AIT  processing  so  that  all  tactical 
actions,  factions  are  consistent  with  the  assigned  role  situation  and 
mission  as  follows: 

a.  Role  situations  are  (1)  peacetime  or  (2)  wartime. 

b.  Basic  missions  are  (1)  transit  or  (2)  antisubmarine  warfare. 
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Figure  12.  Overall  AIT  operation 
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The  major  maneuvering  sequences  available  to  the  AIT,  of  which 
alternative  sets  of  actions  will  occur  on  a  probabilistic  basis,  are: 

a.  Transit  tracks 

b.  Patrol  search  pattern 

c.  Periscope  excursion 

d.  Baffle  clearing  pattern 

e.  Target  detection  events 

f.  Approach  and  torpedo  sequence 

g.  Survi el  lance  sequence 

h.  Torpedo  evasion  and  avoidance  sequence 

i.  Target  evasion  sequence 


3.2.10  TRAINING  SUPPORT  MATERIALS.  A  variety  of  materials  have  been 
developed  to  support  the  tactics  training  process  using  SMARTTS,  as  well  as 
operation  and  maintenance  of  SMARTTS  itself.  These  include  the  standard 
system  operator's  manual,  maintenance  manual,  and  other  supporting 
documentation.  They  also  include  an  operators  course  and  maintenance 
course.  Two  additional  sets  of  support  materials  have  been  developed  to 
better  ensure  effective  use  of  SMARTTS  and  an  effective  tactical  training 
process.  These  consist  of  (a)  an  instructor's  handbook  and  training  course 
aimed  specifically  at  instructional  methodology,  and  (b)  tactics  training 
material  to  support  the  initial  courses  to  be  conducted  under  the  SMARTTS 
configuration. 

An  instructor's  course  and  handbook  were  provided  in  addition  to  the 
standard  operator's  course  for  SMARTTS.  Instructor  personnel  completed  the 
operator's  course  which  addressed  the  operation  of  SMARTTS  in  conjunction 
with  the  21A41A  SCST.  The  instructor's  course  went  the  next  step  in 
addressing  how  to  provide  effective  training  utilizing  the  SMARTTS 
characteristics.  Topics  included  in  the  instructor's  course  and  the 
instructor's  handbook  are: 

a.  Training  process  structure  (Figure  13) 

b.  A  recommended  tactics  training  sequence 

c.  Description  and  application  of  training  techniques 

d.  Exercise  selection 

e.  Specific  tactical  courses 

f.  Conduct  of  preexercise  and  postexercise  briefings 
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Figure  13.  The  training  process 
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g.  Instructor  qualifications 

h.  Instructor  functions 

i.  Learning  principles  applicable  to  team  training 

j.  Team  and  individual  training 

k.  Positive  and  negative  feedback 

l.  Application  of  training  principles 

m.  Training  techniques  and  their  application  (e.g.,  positive  guidance, 
demonstration,  immediate  repetition,  experience  enhancement,  direct 
exposure,  common  error,  team  awareness) 

n.  Appropriate  use  of  SMARTTS  characteristics  during  the  training 
process 


o.  Diagnostic  procedures 

p.  Performance  data  analysis 

q.  Team  self-evaluation 

The  tactics  training  course  material  was  developed  as  a  model  of  the 
application  of  SMARTTS  characteristics  to  the  conduct  of  tactics  training. 
Tactical  exercises  and  supporting  materials  were  developed  for  use  by  Navy 
instructors  to  conduct  tactics  training  using  SMARTTS.  These  materials 
represent  only  the  initial  application  of  SMARTTS.  The  tactics  training 
sequence  in  Figure  14  provides  an  overview  of  the  structure  for  the  tactics 
training  course  material  developed. 

The  intention  of  the  instructor's  course  and  the  tactics  course  was  to 
provide  the  complete  training  system  media  to  achieve  effective  tactics 
training  for  the  initial  application.  It  is,  unfortunately,  atypical  for  an 
instructor's  course  to  actually  address  instruction  rather  than  operation, 
and  for  a  tactics  course  to  be  provided  along  with  the  training  device. 
However,  it  is  believed  that  this  is  certainly  the  most  cost/effective 
approach  since  the  Navy  is  provided  with  an  applied  example  of  the  system's 
application  along  with  support  material,  a  course  and  handbook  on  how  to 
effectively  use  that  system  to  conduct  training,  and  information  addressing 
how  to  evolve  the  system  further.  The  success  of  SMARTTS  and  any  training 
system  depends  primarily  on  the  effectiveness  of  the  instructor.  Not  only 
must  the  instructor  be  effective  in  conducting  the  training  exercises,  but 
also  in  the  development  of  the  exercises,  the  providing  of  feedback,  and  the 
assessment  and  upgrading  of  the  training  program. 
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Figure  14.  training  sequence  chart  -  preprototype  training 
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3.3  SMARTTS  SOFTWARE 


The  SMARTTS  software  consists  of  three  major  components;  (a)  software 
resident  in  the  SMARTTS  host  computer,  the  VAX  11/780;  (b)  SMARTTS  software 
modifications  to  the  Device  21A41  Submarine  Combat  System  Trainer,  Main 
Simulation  Computer  Group  (MSCG);  and  (c)  modifications  to  the  MK117  fire 
control  system  operational  programs  data  gathering  module.  The  major  set  of 
software  is  contained  in  the  SMARTTS  host  computer,  which  performs  the 
majority  of  the  SMARTTS  functions.  Modifications  to  the  existing  MSCG 
software  are  for  the  purposes  of  (a)  collecting  data  from  the  MSCG  (e.g., 
ownship  sonar  parameters);  (b)  to  insert  instructor  commands  from  the  IRC  in 
the  attack  center  (e.g.,  AIT  maneuver  commands);  (c)  to  provide  problem 
control/status  information  to  the  IRC  for  monitoring  by  the  instructor;  and 
(d)  to  enable  automatic  loading  of  the  scenario  from  SMARTTS  to  the  MSCG. 
The  software  modifications  to  the  MK117  fire  control  systems  operational 
software  data  gathering  module  were  made  for  the  purpose  of  collecting  data 
from  the  MK117  for  performance  evaluation  and  feedback  purposes  (e.g., 
target  motion  analysis  system  solution  data).  Whereas,  the  MSCG 
modifications  involved  two-way  communication,  the  MK117  fire  control 
modifications  were  one-way  (i.e.,  data  out  of  the  MK117  to  SMARTTS). 

The  software  structure  ’s  highly  modularized  to  facilitate  maintenance, 
modification,  and  upgrading.  A  detailed  discussion  of  the  SMARTTS  software 
is  beyond  the  intention  of  this  report.  A  brief  summary  of  the  functions 
performed  by  each  of  the  17  SMARTTS  software  modules  is  provided  in  Appendix 
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SECTION  IV 

PRELIMINARY  CONCLUSIONS 


The  initial  SMARTTS  installation  has  been  setup  as  a  preprototype  of 
training  assistance  technology  (TAT),  to  be  evaluated  under  operational 
training  conditions.  The  intention  had  been  to  have  SMARTTS  undergo  a 
6-month  test  and  evaluation  phase,  after  which  a  decision  would  be  made 
regarding  the  training  effectiveness  value  of  each  SMARTTS  characteristic. 
The  exact  form  of  the  SMARTTS  test  and  evaluation  has  not  been  decided  upon 
as  of  this  time. 

The  SMARTTS  preprototype  unit  has  been  installed  on  the  21A41A  submarine 
combat  system  trainer  in  Norfolk,  Virginia.  The  SMARTTS  preprototype  has 
been  used  in  operational  training  exercises  since  its  installation,  and  has 
been  demonstrated  to  various  groups  of  naval  training  personnel.  Since  the 
SMARTTS  evaluation  has  not  yet  taken  place,  it  would  be  premature  to  address 
the  effectiveness  of  SMARTTS  as  a  whole,  or  of  its  individual  capabilities. 
A  summary  of  several  preliminary  observations  are  appropriate  at  this  time, 
however. 

The  training  capability  and  engineering  development  aspects  of  the 
SMARTTS  preprototype  have  been  carried  out  successfully,  as  intended.  The 
preprototype  unit  was  designed,  developed,  and  installed  as  planned.  The 
various  courses  (i.e.,  instructor's  course,  maintenance  course,  operator's 
course)  have  been  conducted.  In  essence,  the  system  has  been  delivered  to 
the  Navy.  SMARTTS  is  currently  being  used,  as  part  of  the  21A41A  SCST,  in 
support  of  a  wide  range  of  submarine  tactics  training.  Most  SMARTTS 
characteristics  have  been  actively  used  to  support  operational  training;  the 
use  of  certain  characteristics  has  depended  on  several  factors  including  the 
group  of  trainees  and  the  particular  exercise  scenarios. 

The  submarine  tactics  training  fleet  project  team  has  observed  SMARTTS, 
and  discussed  its  potential.  Their  preliminary  observation  was  favorable 
towards  the  potential  training  effectiveness  of  the  various  SMARTTS 
characteristics.  This  viewpoint  was  based  on  a  preliminary  observation  of 
the  SMARTTS  characteristics,  and  not  observation  of  their  use  during  the 
training  process.  As  noted  above,  the  fleet  project  team  will  make  a  final 
evaluation  of  the  effectiveness  of  each  SMARTTS  characteristic  after  SMARTTS 
has  been  used  in  training  over  an  appropriate  period  of  time.  Plans  are 
currently  being  developed  for  the  conduct  of  that  evaluation. 

The  capabilities  provided  on  the  preprototype  SMARTTS  have  been 
determined  as  necessary  to  improve  the  cost/effectiveness  of  tactics 
training.  These  capabilities  represent  the  application  of  good  training 
technology,  and  hence  should  be  incorporated  to  a  greater  or  lesser  extent 
on  most  simulator-based  training  devices.  The  early  enthusiasm  which  the 
SMARTTS  preprototype  has  generated  attests  to  the  validity  of  training 
assistance  technology  as  a  necessary  part  of  every  training  device/system. 
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SECTION  V 
RECOMMENDATIONS 


The  SMARTTS  preprototype,  as  a  research  and  development  tool,  provides  a 
testing  ground  for  a  variety  of  training  assistance  technology 
characteristics.  A  variety  of  additional  TAT  characteristics  have  been 
identified,  but  not  included  in  the  preprototype.  Furthermore,  during  the 
development  of  the  SMARTTS  preprototype  and  its  limited  operation  and 
evaluation  at  the  site  other  important  training-related  issues  have  come  to 
light,  many  of  which  might  fall  under  the  guise  of  TAT,  while  others  are 
important  related  issues.  These  issues  are  relevant  to  the  cost/effective 
design  and  operation  of  training  systems.  They  require  a  varying  mix  of 
research  and  development  efforts,  by  the  Navy,  (e.g.,  NTEC)  and  contractors, 
with  investigation  in  the  laboratory  and  in  an  applied  training  setting.  Of 
those  associated  with  TAT,  many  are  implemented  in  the  SMARTTS  preprototype, 
others  are  considered  as  important  candidates  for  upgrading  ( i . e . ,  planned 
evolution)  the  SMARTTS  preprototype,  and  others  are  being  considered  to  be 
added  as  part  of  the  SMARTTS-type  characteristics  that  will  become  part  of 
the  next  generation  of  SCSTs.  Other  issues  may  be  associated  with  SMARTTS, 
but  not  directly  a  part  of  it.  It  is  important  that  these  training-related 
issues  receive  the  necessary  investigative  emphasis,  with  the  continual 
evolution  of  the  training  system  to  continually  achieve  improved  levels  of 
cost /effective  training. 


5.1  RESEARCH  AND  DEVELOPMENT  PLAN  INTRODUCTION 


A  research  and  development  plan  has  been  devised  to  identify  the 
relevant  research  and  development  issues,  and  to  recommend  an  approach  to 
their  solution.  This  research  and  development  plan  is  not  intended  to  be 
the  perfect  approach  to  resolution  of  this  large  variety  of  issues.  Rather, 
its  purpose  is  to  identify  and  organize  the  myriad  of  relevant  issues,  ana 
to  identify  a  plausible  structure  within  which  they  can  be  acceptably 
addressed.  The  number  of  issues  are  vast,  to  the  point  that  it  is  unlikely 
all  can  be  addressed  in  the  near  future.  Hence,  it  is  expected  that" more 
detailed  research  and  development  plans  will  be  developed  to  address 
particular  areas  identified  herein,  and  to  further  delineate  the  specific 
issues  to  be  addressed  under  each  ( i . e . ,  including  time  scheduling,  and  so 
on).  This  research  and  development  plan  is  intended  to  provide  the 
first -cut  high  level  overview  of  an  approach  to  the  continuing  investigation 
of  training  assistance  technology,  and  its  associated  issues  with  regard  to 
the  development  of  cost  effective  training  systems. 

Several  fundamental  dimensions  pertain  to  the  organization  of  the  plan, 
and  how  it  might  be  carried  out.  The  brief  discussion  of  these  dimensions, 
and  the  following  detailed  recommendations,  has  been  developed  to  provide 
information  regarding  this  author's  opinions  concerning  the  organization  anc 
implementation*  of  a  research  and  development  plan  addressing  Training 
Assistance  Technology.  An  extensive  analysis  of  current  training  needs  and 
other  relevant  considerations  has  not  been  formally  performed.  Rather,  the 
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recommendations  provided  herein  reflect  the  experiences  and  background 
associated  with  TAT,  SMARTTS,  and  other  training  research  and  development 
work.  It  is  expected,  therefore,  that  these  recommendations  will  form  one 
of  several  information  sources  used  to  evolve  and  update  current  research 
and  development  plans.  The  fundamental  dimensions  of  the  plan  are 
identified  below  and  discussed  in  the  following  paragraphs: 

a.  Work  area  —  Three  major  areas  have  been  identified:  (1)  21A41A  site 
work,  (2)  training  assistance  technology  work,  and  (3)  other  related  work. 

b.  Work  type  —  Basic  research,  applied  research,  development  to  refine 
existing  capabilities,  development  to  add  new  capabilities. 

c.  Location  —  Site  work  to  be  performed  at  appropriate  operational 
training  sites;  (e.g.,  21A41A  SCST)  and  laboratory  work  to  be  performed  in  a 
laboratory  setting,  with  an  appropriate  apparatus  configuration  (e.g..  Navy 
laboratory;  contractor's  laboratory). 

d.  Application  —  Direct  training  process  function,  or  training  support 
function. 

The  work  area  was  selected  as  the  primary  dimension  for  organization  of 
the  research  and  development  plan.  The  preprototype  SMARTTS  installation  on 
the  21A41A  SCST  provides  an  onsite  research  and  development  capability  which 
has  the  potential  of  providing  extremely  valuable  information  regarding  the 
cost/effectiveness  of  training  system  characteristics.  Each  time  a  training 
exercise  is  conducted  at  the  site,  empirical  data  is  generated  regarding  the 
use  of  various  SMARTTS  characteristics,  the  effectiveness  of  the  training 
exercise,  and  so  on.  It  is  important  that  the  Navy  take  advantage  of  this 
extremely  important  information.  Onsite  research  has  an  extremely  important 
role  in  the  R&D  process.  Since  it  can  provide  the  end-product  data  for 
evaluation  of  training  system  characteristics  in  the  applied  training 
setting.  Hence,  the  first  major  segment  of  the  research  and  development 
plan  addresses  the  onsite  work.  The  second  work  area,  TAT,  addresses  a 
variety  of  characteristics  identified  within  the  SMARTTS  program.  Many  of 
these  characteristics  are  included  in  the  preprototype  SMARTTS,  while  others 
have  been  identified  but  not  yet  implemented.  This  second  work  area 
category  of  research  and  development  addresses  those  characteristics 
considered  as  a  part  of  TAT.  The  final  work  area  category  (other  related 
work)  addresses  a  variety  of  important  training  issues  not  in  the  other  two 
categories.  The  technical  elements  of  the  research  and  development  plan  are 
addressed  under  each  of  these  three  major  work  areas,  comprising  the  bulk  of 
the  plan. 

The  work  type  to  be  performed  is  an  important  secondary  dimension.  It 
has  a  direct  impact  on  the  priority  of  work  to  be  performed,  with 
considerations  for  the  time  frame  to  conduct  the  work,  the  selection  of  who 
should  perform  the  work  (e.g..  Navy  laboratory,  contractor).  It  also 
impacts  where  the  work  should  be  performed,  at  the  site  or  in  a  laboratory, 
the  type  of  personnel,  and  funding  sources. 

The  location  dimension  addresses  the  recommended  setting,  for  performing 
the  work,  at  the  training  site  or  in  the  laboratory.  This  dimension 


64 


NAVTRAEQUIPCEN  80-00079 


overlaps,  to  some  extent,  with  the  work  area  dimension  identified  above  and 
with  the  other  dimensions.  However,  it  also  provides  indepenaent 
information.  The  location  is  particularly  sensitive  to  the  work  type. 
Basic  research  should  usually  be  performed  in  the  laboratory;  it  should 
identify  those  TAT  concepts  and  characteristics  that  have  potentially  high 
training  value.  Applied  research  should  also  be  performed  in  the 
laboratory,  when  cost-effective,  so  as  not  to  interfere  with  operational 
training.  Often,  however,  the  requisite  applied  research  can  only  be 
conducted  at  the  operational  site.  Development  work,  like  applied  research 
may  be  best  accomplished  in  both  locations. 

Careful  consideration  must  be  given  to  the  performing  organization 
(Navy,  contractor).  This  dimension  generally  focuses  on  the  resources 
available  to  perform  the  necessary  work  in  a  timely  and  cost/effective 
manner.  The  three  above  dimensions  have  a  direct  impact  on  the  selection  of 
the  performing  organization.  Navy  laboratories  and  contractors  each  have 
unique  capabilities  and  limitations,  which  should  be  carefully  considered  in 
determining  the  performing  organization.  Factors  to  be  considered  include: 
(a)  interface  with  the  operating  forces  (e.g.,  operational  experience  of 
team  members;  protocol);  (b)  resident  basic  and  applied  research  skills,  and 
previous  experience;  (c)  subject  matter  expertise;  (d)  proximity  to  data 
collection  sites,  including  accessibility  to  necessary  technical 
information;  and  (e)  specific  prior  work  (e.g.,  the  SMARTTS  system  developer 
should  perform  the  system  modifications).  It  is  beyond  the  scope  of  this 
research  and  development  plan  to  recommend  performing  organizations  (even 
generically)  for  the  various  elements.  Hence,  this  dimension  will  not  be 
addressed  further  in  this  plan. 

The  products  emanating  from  work  addressing  elements  of  this  plan  may  be 
directly  related  to  the  conduct  of  the  training  process  (e.g.,  feedback 
displays),  or  may  be  of  an  indirect  training  process  support  nature  (e.g., 
long-term  analysis  capabilities).  This  application  has  an  impact  on  the 
priority  allocated  to  the  various  elements  of  this  plan.  Generally,  those 
elements  directly  impacting  the  training  process  have  been  given  a  higner 
priority. 

The  work  area  represents  the  primary  dimension  for  addressing  the 
elements  of  this  plan.  The  other  three  dimensions  are  secondary,  although 
also  relevant  to  categorization  of  the  work.  The  research  and  development 
plan  has  two  major  parts:  (a)  identification  of  research  and  development 
issues  (Section  5.2)  and  (b)  a  research  and  development  plan  summary 
(Section  5.3)  .  The  first  part  describes,  in  some  detail,  each  of  the  101 
recommended  issues.  This  part  groups  the  issues  under  the  three  work 
areas.  The  plan  structure  is  summarized  in  Figure  15,  showing  each  of  the 
three  work  areas,  and  their  major  issue  categories  respectively.  The 
discussion  in  this  first  part  addresses  the  issues  under  each  of  these 
categories.  This  is  the  most  essential  part  of  the  plan  since  it  identifies 
all  the  research  and  development  issues.  The  second  part  of  the  plan 
provides  information  about  each  of  the  other  dimensions,  pertaining  to  each 
issue  (e.g.,  priority,  estimated  cost).  This  information  is  mostly 
presented  in  tabular  form  in  Table  1,  and  discussed  in  the  research  and 
development  plan  summary  contained  in  Section  5.3. 
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5.2  RESEARCH  AND  DEVELOPMENT  ISSUES 


Recommended  Research  and  development  issues  are  identified  and  described 
below  under  the  major  work  area  categories  of  21A41A  site  work  (Section 
5.2.1),  TAT  research  and  development  (Section  5.2.2),  and  other  areas 
(Section  5.2.3).  Each  work  area  is  organized  into  several  major  subareas 
pertaining  to  specific  TAT  capabilities  investigative  issues.  Specific 
elements  of  work  are  addressed  under  each  subarea. 

Note,  each  of  the  101  research  and  development  issues  is  numbered 
consecutively  in  the  order  presented  in  this  plan.  This  number  provides 
correspondence  with  the  listing  of  issues  in  Table  1,  which  provides 
information  relevant  to  the  other  fundamental  dimensions  of  part  of  this 
plan  (note.  Table  1  is  discussed  in  the  subsequent  part  of  this  section. 
Research  and  Development  Plan  Summary,  Section  5.3). 


5.2.1  21A41A  SITE  WORK.  The  installation  of  the  SMARTTS  preprototype  as 
part  of  the  21A41A  provides  the  Navy  and  NTEC  with  an  excellent  opportunity 
to  conduct  empirical  research  in  the  applied  training  environment.  The 
initial  plans  during  SMARTTS  development  were  to  install  a  preprototype 
SMARTTS  as  part  of  the  21A41A,  to  evaluate  the  effectiveness  of  each  of  the 
SMARTTS  characteristics,  and  to  modify  the  characteristics  as  necessary 
leading  to  the  design  specification  for  the  SMARTTS  prototype  unit.  This 
thrust  of  work  remains  a  major  recommended  part  of  the  work  to  be  performed 
under  this  area.  Other  work  is  also  recommended  to  be  performed  under  this 
area  in  conformance  with  need  and  site  availability. 

5. 2. 1.1  Preprototype  Test  and  Evaluation.  (1)  The  preprototype  SMARTTS 
should  be  evaluated  during  operational  training.  Aspects  of  concern  during 
this  evaluation  are: 

a.  The  overall  TAT  concept  as  exemplified  by  SMARTTS  in  a  generic  sense. 

b.  Each  specific  SMARTTS  characteristic: 

As  currently  implemented  on  the  21A41A. 

With  regard  to  its  impact  on  submarine  tactics  training 
effectiveness. 

With  regard  to  its  likely  impact  on  training  effectiveness  in 
other  applications. 

c.  SMARTTS  characteristics  should  be  modified  as  necessary  during  the 
evaluation  period  to  insure  that  the  evaluation  considers  the  true  potential 
of  each  characteristic,  rather  than  being  constrained  by  the  particular 
manner  in  which  it  has  been  implemented  in  this  initial  application. 

d.  The  opinions  of  the  instructional  staff  along  with  objective 
evaluations  should  constitute  the  data  base  for  the  investigation. 

e.  The  evaluation  should  be  conducted  over  a  period  of  months  to  insure 
that  the  instructional  staff  has  adequate  familiarization  with  the 
application  of  the  various  SMARTTS  characteristics. 
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f.  The  amount  of  utilization  of  each  SMARTTS  characteristic  should  be 
assessed  (e.g.,  number  of  times  used  per  exercise,  total  number  of  minutes 
used  per  exercise).  This  should  form  one  source  of  information  for 
evaluation  purposes,  and  to  direct  modification  efforts. 

g.  As  rigorous  a  methodological  approach  as  possiDle  should  be  followed 
for  this  evaluation. 


5. 2. 1.2  Refine  SMARTTS.  (2)  The  SMARTTS  characteristics  are  largely 
implemented  via  the  man-machine  interface,  for  which  relatively  little 
objective  data  has  been  available  to  guide  design  practices.  Although  the 
generic  SMARTTS  characteristics  stem  from  an  extensive  training  analysis, 
the  specific  interface  characteristics  (e.g.,  specific  display  formats)  have 
been  largely  organized  on  a  basis  of  "good  human  factors  design".  They  have 
not,  however,  benefitted  from  an  in-depth  human  factors  analysis  regarding 
the  particular  operational  sequences,  etc.  It  is  expected,  therefore,  that 
substantial  improvements  in  the  interface  characteristics  can  be  made,  as 
experience  is  gained,  to  substantially  improve  instructor  functioning.  An 
example  is  the  command  sequences  for  various  functions  on  the  plasma  entry 
device.  If  appropriate  empirical  data  were  available  regarding  the 
particular  sequences  that  the  instructor  is  likely  to  use,  a  more  effective 
design  could  be  achieved.  It  is  recommended,  therefore,  that  an 
investigation  be  conducted  to  identify  specific  aspects  of  SMARTTS  that 
would  benefit  from  refinement  of  the  man-machine  interface.  Several 
specific  research  elements  are: 

a.  (3)  Refinement  of  the  cursor  control  on  the  displays,  such  as  the 
trigraph  and  time  function  plots. 

b.  (4)  Provision  of  a  visual  indication  for  the  audible  plasma  signal. 

c.  (5)  The  interactive  sequence  with  the  plasma  entry  device  may  be 
substantially  improved  if  it  is  tailored  to  the  likely  actions  of  the 
instructor.  For  example,  those  operations  most  often  performed  by  the 
instructor  should  have  the  shortest  series  of  plasma  entry  steps.  Data  for 
this  analysis  could  be  conducted  from  discussion  with  the  instructors;  also, 
an  automated  routine  could  be  inserted  into  the  SMARTTS  software  to 
automatically  monitor  the  sequence  of  input  commands  made  by  instructors. 
The  entry  sequence  could  be  reformatted  to  minimize  the  number  of  steps 
required  overal 1 . 

d.  (6)  Plasma  entry  sequence  should  be  modified  to  allow  the 
instructor/operator  to  manipulate  the  scaling  parameters  of  the  approach 
geodisplay  format  (i.e,  range  scale,  and  plot  center)  without  having  to 
re-enter  the  command  tree  each  time.  This  would  greatly  facilitate 
manipulation  of  the  display  by  the  instructor  to  achieve  the  desired  picture. 

e.  (7)  Evaluate  the  system  response  time  for  various  entry  commands  to 
identify  those  entry  commands  that  take  too  much  time.  The  entry  routine 
should  be  modified  to  speed  up  the  system  response  to  these  commands. 
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f.  (8)  Investigate  a  means  of  identifying  each  target  on  the  approach 
geoplot  display  (e.g.,  joystick  for  positioning  of  a  cursor  over  a  target) 
and  obtaining  an  alphanumeric  readout  of  the  target  characteristics. 

g.  (9)  Investigate  the  use  of  the  alternative  tactics  characteristics 
from  an  interface  standpoint,  leading  to  redesign  of  its  operation.  The 
alternative  tactics  feature  as  currently  implemented  appears  to  be  somewhat 
time  consuming.  Partially  automating  this  capability  may  help,  such  as 
providing  three  programmed  alternatives  associated  with  each  library 
exercise.  Other  ways  of  manipulating  the  CRT  plot  (e.g.,  allowing  the 
instructor  to  simply  point  to  the  position  he  wants  ownship  to  move  to,  and 
having  the  computer  calculate  the  necessary  course  and  speed  change  and 
other  variables)  should  be  investigated.  Note,  the  alternative  taccics 
capability  is  also  identified  under  additional  TAT  capabilities  relevant  to 
site  work  (Section  5. 2. 1.3,  #17),  and  addressed  in  greater  detail  under  TAT 
research  and  development,  alternative  actions  (Section  5. 2. 2. 2,  #57). 

h.  (10)  The  actual  ownship  action  taken  in  a  particular  problem  should 
be  automatically  presented  as  one  alternative  when  engaged  in  an  alternative 
tactics  investigation.  This  would  enable  the  other  selected  alternative 
actions  to  be  compared  with  the  actual  tactical  action  followed  by  ownship 
without  the  necessity  of  manual  entry  by  the  instructor. 

i.  (11)  Investigate  the  use  of  color  on  the  information  displays  to 
determine  the  most  effective  principles  of  application.  For  example, 
perhaps  use  color  to  distinguish  the  difference  between  friendly  and  hostile 
forces.  Also,  the  use  of  color  may  differ  for  various  applications,  such  as 
monitoring  and  control  of  the  training  exercise  scenario  versus  presentation 
of  feedback  information  in  the  classroom. 

j.  (12)  When  a  new  target  of  interest  is  selected,  the  current  display 
formats  being  presented  should  automatically  change  to  present  the 
appropriate  information  relevant  to  the  new  target.  Re-entry  of  the  command 
sequence  is  currently  necessary  to  bring  up  the  display  format  for  the  new 
target. 


k.  (13)  Add  the  capability  to  shift  ownship  to  various  vessels  in  the 
exercise  scenario  from  the  instructor's  console. 

l.  (14)  Add  a  "hard  copy"  switch  on  the  plasma  such  that  the  instructor 
can  remotely  activate  the  hard  copy  unit  to  copy  the  CRT  image. 

m.  (15)  Automatically  enter  the  subjective  performance  indicator  when 
selected  by  the  instructor.  At  present,  the  instructor  must  push  an 
additional  entry  button  after  he  has  indicated  the  performance  indicator 
selected. 

n.  (16)  An  investigation  should  be  conducted  to  determine  if  the 
performance  indicators  CRILS  and  CRALS  should  be  referenced  to  the  selected 
target  position  or  the  predetermined  firing  point  for  that  particular 
target.  Differences  of  opinion  currently  exist. 
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5. 2. 1.3  Add  TAT  Capabilities.  (17)  Modification  of  the  existing 
preprototype  SMARTTS  characteristics  should  be  accomplished  to  overcome 
current  inadequacies,  and  to  further  evolve  the  system  capabilities.  Each 
of  these  modification  classes  will  result  in  a  more  effective  training 
system.  Whereas  the  previous  issue  (i.e.,  refinement)  addresses  the  tuning 
of  the  preprototype  characteristics,  this  issue  addresses  the  addition  of 
new  characteristics  to  the  preprototype.  Obviously,  there  may  be 
considerable  interpretation  with  regard  to  refinement  versus  addition  of  new 
characteristics.  Refinement,  as  used  above,  has  generally  meant  the 
improvement  of  existing  characteristics,  while  this  section  is  addressing 
the  implementation  of  additional  characteristics.  The  recommended  elements 
are  as  follows: 

a.  (18)  A  capability  should  be  added  to  facilitate  the  modification  of 
Pis  and  the  addition  of  new  Pis  by  instructor  personnel.  Although  the 
SMARTTS  software  structure  facilitates  modification  this  can  currently  be 
achieved  only  if  the  instructor  is  well  versed  in  the  SMARTTS  software  and 
its  modification.  One  means  of  providing  the  instructors  with  this 
capability  is  via  an  instructor  interface  language,  which  is  discussed  in 
more  detail  under  the  TAT  area,  instructor  specific  (#82).  This  capability 
should  be  provided  to  instructor  personnel  via  the  model  deck  VT100  terminal. 

b.  (19)  Additional  Pis  should  be  developed  and  added  to  the  SMARTTS 
repertoire.  A  major  aspect  of  SMARTTS  during  the  design  was  its  capability 
to  evolve  as  the  training  situation  evolved.  Pis  should  be  modified  and/or 
added  as  necessary  to  support  effective  tactics  training.  The  research  and 
development  of  Pis  is  both  a  site  issue  and  a  TAT  research  and  development 
issue.  Several  specific  PI  recommendations  have  been  made  by  21A41A  site 
personnel;  these  should  be  readily  implemented.  In  addition,  a  long  term 
research  and  development  effort  should  be  initiated  for  the  continual 
development  and  upgrading  of  Pis;  this  is  discussed  later  under  the  TAT 
area.  Performance  Indicators,  Section  5. 2. 2. 3,  #58.  Specific  Pis  that  have 
been  suggested,  and  are  recommended  to  be  implemented  at  the  site,  include: 


Towed  array  stabilization  times  as  a  function  of  speed 
MK48  torpedo  detection/acquisition  range 
Probability  of  weapon  hit 

MK48  presets  --  tactical  variables  rather  than  Pis 


c.  (20)  The  tactical  effectiveness  factor  (TEF)  should  be  investigated 
to  determine  if  it  can  be  implemented  via  SMARTTS.  The  TEF  is  used  by 
COMSUBLANT  to  provide  an  estimate  of  a  ship's  tactical  ability,  based  on 
observation  of  at-sea  exercise  performance  across  different  units.  The  TEF 
parameter  may  provide  relevant  information  during  training  since  several 
performance  indicator-type  parameters  are  associated  with  the  calculation  of 
TEF.  Each  of  these  individual  parameters  may  also  be  relevant  as  Pis  during 
the  training  exercise,  and  should  thus  be  investigated. 


d.  (21)  A  time/frequency  plot  should  be  developed  for  presentation  of 
feedback  information.  The  information  that  would  be  addressed  on  this  plot 
is  directly  relevant  to  an  important  aspect  of  tactics  training.  The 
information  necessary  to  construct  the  plot  is  readily  available  in  the 
simulation  computer. 
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e.  (22)  The  geoplot  feedback  display  shc:ld  be  investigated  for 
modification,  or  for  the  development  of  an  additional  related  display.  The 
current  geoplot  feedback  display  appears  effective  as  currently  configured. 
Hence,  it  may  be  desirable  to  develop  a  similar  display  with  the  additional 
necessary  information  to  further  support  the  tactical  training  objectives 
and  provide  necessary  feedback.  Recommended  modifications  include: 

Extend  bearing  lines  beyond  the  actual  target,  showing  maximum 
range  for  estimated  speed. 

Superimpose  the  actual  bearing  lines  on  the  display  ( i . e . ,  buzz 
bearings  and  automatic  target  following  bearings). 

Provide  indication  on  the  display  for  the  minimum/maximum 
ranges,  Ekelund  ranges  when  determined,  and  other  range  related  solution 
data. 


Additional  parameters  may  also  be  added  to  this  display  format, 
including  readouts  addressing  target  range,  range  rate,  and  line-of-sight 
information. 

f.  (23)  The  applicability  of  an  information  display  format  showing 
multiple  line-of-sight  diagrams  to  several  targets  should  be  investigated. 
This  display  format  would  present  up  to  four  line-of-sight  diagrams  from 
ownship  to  selected  targets.  The  line-of-sight  diagram  is  an  integral  tool 
used  during  submarine  tactical  engagements  to  evaluate  the  geometrical 
relationships  between  ownship  and  the  various  targets.  Multiple 
line-of-sight  diagrams  would  be  useful  during  an  encounter  with  several 
targets  simultaneously. 

g.  (24)  The  capability  should  be  added  to  enable  the  designation  of  any 
target  as  the  source  of  information  for  generating  the  display  formats. 
That  is,  as  currently  configured,  the  line-of-sight  diagram.  Pis,  and 
tactical  variables  presented  on  the  display  formats  pertain  to  only  the 
primary  target.  The  capability  should  be  provided  in  SMARTTS  to  enable  the 
instructor  to  designate  any  target  as  the  source  of  this  information. 
Hence,  the  instructor,  and/or  trainees,  could  view  the  various  display 
formats  and  information  summaries  as  pertaining  to  any  of  the  targets 
included  in  the  problem. 

h.  (25)  The  effectiveness  of  providing  a  line-of-sight  diagram  from 
ownship's  weapon  (e.g.,  torpedo)  to  the  target  should  be  investigated.  It 
appears  that  this  display  format  may  be  a  useful  training  aid  with  regard  to 
controlling  the  weapon  postlaunch. 

i.  (26)  An  investigation  should  be  initiated  to  develop  a  time/range 
plot  providing  relevant  summary  information  relating  to  the  system  solution, 
the  system  solution  range  error  (i.e.,  PI),  and  the  actual  range.  The 
display  format  should  be  developed  to  integrate  these  important  tactical 
variables  and  Pis  to  present  an  effective  feedback  picture  relevant  to 
evaluation  of  the  amount  and  direction  of  target  motion  analysis  errors,  and 
range  relevant  to  the  firing  point. 
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j.  (27)  A  signal  to  noise  ratio  (SNR)  versus  time  plot  linked  to  the 
different  modes  of  the  AN/BQQ5  sonar  operation,  should  be  investigated  as  an 
addition  to  SMARTTS  for  feedback  purposes.  This  plot  would  specifically 
pertain  to  the  50%  probability  of  detection  SNR  limits.  The  format  would 
present  a  plot  of  actual  SNR  over  time  of  the  exercise,  with  the  relevant 
recognition  differential  levels  indicated. 

k.  (28)  The  symbol  repertoire  of  the  SMARTTS  displays  should  be 
expanded  to  include  additional  NTDS  symbology  (e.g.,  friendly  and  target 
helicopters,  sonar  buoys,  and  decoys). 

l.  (29)  A  feedback  display  should  be  developed  to  present  the  torpedo 
vertical  and  horizontal  plane  search  position  versus  the  target's  position. 
This  display  would  provide  information  to  the  trainees  regarding  the  torpedo 
track  in  the  three  dimensional  environment,  and  its  acquisition  of  the 
target.  Several  relevant  display  concepts  are  presented  later  under  the  TAT 
area,  information  presentation  (Section  5. 2. 2. 4,  #65),  that  address  the  use 
of  three  dimensional  displays. 

m.  (30)  A  depth  option  should  be  added  to  the  alternative  tactics 
capability,  to  permit  examination  of  the  impact  of  changing  ownship  depth 
relative  to  the  target  and  the  environment. 

n.  (31)  The  alternative  tactics  characteristics  are  believed  to  be  an 
extremely  valuable  tool  to  aid  in  the  learning  of  complex  relationships. 
The  alternative  tactics  setup  procedures  used  on  SMARTTS,  however,  are 
somewhat  laborious  as  compared  with  other  SMARTTS  operational  procedures. 
It  is  recommended  that  an  investigation  be  initiated  to  evaluate  various 
approaches  to  the  implementation  of  alternative  tactics  setup  and  control 
procedures.  For  example,  several  relevant  alternative  ownship  and  target 
actions  could  be  predetermined  and  stored  on  the  training  device;  these 
could  be  automatically  generated  via  a  single  button  push  command  by  the 
instructor.  Hence,  the  instructor  would  have  rapid  access  to  several 
relevant  actions  pertaining  to  each  problem.  He  would  also  be  able  to 
manually  enter  other  alternatives  as  he  so  desired.  Other  types  of 
instructor  interface  characteristics  should  be  investigated  in  this  regard 
also.  Note,  alternative  tactics  is  also  identified  under  refine  SMARTTS, 
relevant  to  site  work  (Section  5. 2. 1.2,  #2),  and  addressed  in  greater  detail 
under  TAT  research  and  development,  alternative  actions  (Section  5. 2. 2. 2, 
#57) . 


o.  (32)  Modifications  to  the  approach/geoplot/1 ine-of-sight  display 
should  be  investigated  with  regard  to  the  inclusion  of  periscope-related 
information.  The  trainees  obtain  certain  information  during  the  periscope 
observation  that  can  be  compared  with  the  similar  computer-generated  (i.e., 
actual)  information.  The  relevant  parameters  include  target  bearing,  target 
angle-on-the-bow,  target  range,  and  time.  The  actual  computer-generated 
information  can  be  provided  on  a  display  for  comparison  with  that 
information  manually  collected  by  the  trainee,  and  recorded  by  the 
instructor.  This  information  could  be  added  to  the  approach/geoplot  display. 

p.  (33)  Additional  subjective  Pis  should  be  investigated  relevant  to 
periscope  operation,  following  from  the  above  recommendation.  During  a 
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periscope  observation  the  instructor  could  manually  enter  subjective 
information  into  SMARTTS  via  the  instructor's  console,  based  on  the  trainees 
observations.  A  comparison  of  the  actual  and  the  student-generated  values 
could  then  be  provided  as  relevant  feedback  information  via  one  or  more 
SMARTTS  display  formats.  This  is  considered  as  an  important  aspect  in 
tactical  training  (i.e.,  periscope  operation).  Pis  should  be  developed  and 
tactical  variable  information  collected  for  airborne  and  missile  platforms. 

q.  (34)  Additional  data  should  be  collected  from  the  MK117  fire  control 
system  pertaining  to  the  actions  of  individual  operators.  Data  relevant  to 
the  editing  actions  of  the  MK81  weapon  control  console  operator,  for 
example,  would  provide  relevant  feedback  information.  SMARTTS  generally 
addresses  the  overall  team  and  ship  performance,  and  to  a  lesser  extent  the 
actions  and  performance  of  individuals.  SMARTTS  should  be  expanded  to 
address  in  greater  detail  the  actions  of  individual  operators  such  that  it 
can  further  assist  the  instructor  in  providing  a  more  effective  training 
process  in  this  regard.  This  expansion  should  focus  around  the  collection 
of  appropriate  data,  the  development  of  necessary  performance  indicators, 
and  the  development  of  appropriate  display  formats  for  monitoring  by  the 
instructor  and  for  providing  feedback  and  instructional  guidance. 

r.  (35)  The  training  material  developed  as  a  part  of  the  preprototype 
SMARTTS  should  be  expanded  to  provide  greater  coverage  of  submarine  tactical 
training  requirements.  That  is,  the  exercise  library  should  be  expanded 
along  with  the  appropriate  instructor  and  trainee  guides  to  address  a 
greater  breadth  of  tactics  training  (e.g.,  over  the  horizon  missile 
capability).  The  training  material  currently  provided  via  SMARTTS  addresses 
only  the  basic  level  of  tactics  training;  intermediate  and  advanced  level 
training  material  should  be  developed. 

s.  (36)  Additional  relevant  tactical  variable  data  should  be  collected 
from  the  MKU7  fire  control  system.  SMARTTS  currently  collects  and  presents 
as  feedback  information  data  relevant  to  target  motion  analysis  (e.g., 
system  solution  accuracy).  A  variety  of  additional  data  are  available  for 
collection  and  presentation  from  the  MK117  FCS.  These  include,  for  example, 
MK48  weapon  presets.  An  analysis  should  be  made  of  the  complete  set  of 
relevant  tactical  variable  data  to  be  collected  from  the  MK117  FCS,  and 
SMARTTS  modified  to  effect  that  collection.  Appropriate  feedback  display 
formats  would  have  to  be  developed  to  present  the  information  as  appropriate 
relevant  to  the  various  training  exercises. 

t.  (37)  The  approach  geoplot/1 ine-of-sight  display  should  be 
investigated  with  regards  to  modification  for  inclusion  of  additional 
relevant  information.  This  display  format  has  been  used  extensively  during 
training  to  date.  It  should  be  investigated  for  inclusion  of  additional 
tactical  variables  and  Pis.  For  example,  probability  of  counterdetection 
and  probability  of  detection  rings  could  be  placed  on  the  display  format  as 
an  additional  source  of  information  during  exercise  monitoring  by  the 
instructor,  as  well  as  for  use  during  postproblem  feedback  sessions.  Other 
relevant  information,  for  example,  should  be  investigated  for  inclusion  on 
the  geoplot  display  relevant  to  detection  (e.g.,  detection  limits  for  bottom 
bounce,  direct  path,  and  conversion  zone). 
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u.  (38)  A  variety  of  AIT  status  information  is  provided  via  SMARTTS 
displays.  However,  the  AIT  operation  is  complex,  both  in  a  real-time  sense 
and  from  a  logic/decision  structure  standpoint.  The  type,  amount,  and 
timing  of  AIT  related  information  to  be  presented  to  the  instructor  should 
be  investigated.  Additional  information  relevant  to  (a)  impending  AIT 
actions,  (b)  the  logic  buildup  leading  to  the  selection  of  those  actions, 
and  (c)  the  options  available  to  the  AIT  and  forthcoming  logic  paths  may  be 
appropriate  for  display  to  the  instructor  and/or  the  trainees.  AIT-related 
information  is  important  from  several  standpoints:  (a)  the  instructor 
requires  a  certain  amount  of  information  to  oversee  operation  of  the  AIT, 
and  directly  control  it  if  necessary;  and  (b)  the  logical  operation  of  the 
AIT  is  instructive  in  itself,  since  it  represents  the  composite  structure  of 
the  best  available  intelligence  information  pertaining  to  the  particular 
target.  The  SMARTTS  information  displays  substantially  address  the  first  of 
these  AIT  uses;  the  adequacy  of  the  information  in  this  regard  should  be 
investigated,  and  modified  as  necessary.  The  second  use  of  the  AIT  has  not 
yet  been  addressed  during  training,  although  it  is  believed  that  the  AIT 
would  be  an  excellent  tool  for  demonstrating  actions  of  the  various  targets 
modeled.  This  aspect  is  discussed  later  under  the  TAT  area,  AIT  (Section 
5.2.2. 1,  #51).  In  either  case,  investigation  should  proceed  to  determine 
the  information  display  requirements  relevant  to  use  of  the  AIT. 

v.  (39)  The  SMARTTS  instructor  consoles  (IRC  and  CRC)  provide  a  primary 
color  graphics  CRT  display  surface,  and  a  plasma  entry  device  that  has  the 
capability  of  use  as  a  secondary  display  surface.  Investigation  should  be 
initiated  to  determine  if  the  need  exists  for  an  additional  primary  display 
surface  added  to  the  IRC  and/or  CRC.  An  additional  instructor  console 
display  would  allow  the  instructor  to  maintain  status  information  on  one 
display  while  investigating  other  relevant  aspects  on  another  display.  The 
additional  display  surface  could  be  added  to  the  existing  instructor 
consoles.  The  necessity  for  such  a  display  is  unclear  at  this  time,  hence 
an  investigation  is  suggested  to  determine  the  effectiveness  and  feasibility 
of  an  additional  display  surface. 

w.  (40)  An  additional  display  surface  is  recommended  for  information 
presentation  in  the  classroom,  in  addition  to  the  large  screen  display.  A 
major  aspect  of  SMARTTS  is  its  ability  to  provide  information  feedback  (and 
prebriefing  information)  in  visual  formats  to  the  trainees  in  the 
classroom.  The  need  currently  exists  to  simultaneously  view  several  display 
surfaces,  at  times  (e.g.,  approach  geoplot  display  to  provide  a  picture  of 
the  evolving  geometrical  situation,  together  with  a  trigraph  display  of  Pi's 
and  tactical  variables).  The  SMARTTS  display  formats  have  been  designed  to 
provide  as  much  relevant  information  as  possible  on  a  single  display 
surface,  without  overloading  the  visual  presentation  (i.e.,  in  some  cases 
such  as  the  trigraph  display  the  total  amount  of  information  available  for 
presentation,  if  selected  by  the  instructor,  may  exceed  the  trainee's  limit 
to  process).  A  second  display  surface  in  tne  classroom  would  enable 
additional  information  to  be  provided  in  a  clear  format.  Hence,  an 
additional  display  surface  should  be  added. 

x.  (41)  A  display  format  should  be  developed  to  present  information 
regarding  the  actual  hand-computed  Ekelund  range  solutions.  That  is,  a 
computer  generated  Ekelund  range  solution  based  on  the  actual  target 
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parameters,  and  sonar-received  target  parameters  should  be  generated  for 
comparison  with  the  manual  Ekelund  ranges  generated  by  the  fire  control 
party.  An  appropriate  feedback  display  format  should  be  developed  for 
presentation  of  this  information,  or  it  could  be  added  to  the  existing 
geoplot  display  format. 

y.  (42)  A  comprehensive  freeze  capability  should  be  added  to  the 
21A41A.  SMARTTS  has  the  capability  to  freeze.  The  remaining  subsystems  of 
the  21A41A  (i.e.,  MSCG,  MK117,  and  21B64)  can  not  completely  freeze  the 
problem.  The  capability  of  a  freeze  has  been  discussed  for  some  time  by  the 
various  groups  concerned  with  submarine  tactics  training.  This 
recommendation  is  currently  being  worked  on  by  NTEC  and  other  groups  within 
the  submarine  training  community. 

z.  (43)  A  weapon  model  capability  should  be  added  to  SMARTTS  to  provide 
for  classroom/standalone  generation  of  weapon  actions.  In  the  SMARTTS 
standalone  configuration  (i.e.,  during  prebriefing,  in  the  classroom,  etc.) 
the  SMARTTS  models  can  run  an  exercise  problem  in  fast-time  or  real-time  up 
until  the  point  of  fire,  generating  appropriate  tactical  variables  and 
performance  indicators.  It  is  recommended  that  this  capability  be  expanded 
to  include  weapon  firing,  postlaunch  weapon  control,  evasion,  and  so  on.  It 
should  be  noted  that  SMARTTS  does  address  weapon  actions  that  took  place  in 
the  real-time  context  during  the  attack  center  exercise  (i.e.,  SMARTTS  can 
provide  feedback  information  relative  to  weapon  actions).  However,  SMARTTS 
cannot  address  alternative  tactics  regarding  weapon  actions,  nor  address 
weapon  actions  in  the  standalone  mode. 

aa.  (44)  The  Tomahawk  and  Harpoon  capability  should  be  added  to 
SMARTTS.  This  would  include  the  weapon  modeling  with  regard  to  standalone 
operation  as  noted  above,  and  also  the  development  of  relevant  Pis,  tactical 
variables,  and  feedback  display  formats.  Employment  of  the  Tomahawk  ano 
Harpoon  will  become  a  major  focal  point  of  tactics  training  in  the 
relatively  near  future.  Since  Tomahawk  and  Harpoon  represent  an 
evolutionary  step  in  the  complexity  of  submarine  tactics  it  is  especially 
appropriate  for  SMARTTS  training  support. 

bb.  (45)  The  SMARTTS  sonar  and  environmental  modeling  capabilities 
should  be  expanded  to  include  other  relevant  aspects  (e.g.,  convergence 
zone,  the  ability  to  select  and  simulate  the  various  standard  oceans,  ocean 
layers,  Lloyd-mirror  effect  and  bottom  bounce).  The  current  models  provide 
basic  sonar  and  environmental  capabilities;  these  should  be  improved  to 
enable  SMARTTS  to  address  a  wider  range  of  training  needs. 

cc.  (46)  The  capability  should  be  developed  to  model  additional  target 
weapons  to  be  fired  against  ownship.  Similar  to  the  need  for  providing  for 
advanced  ownship  weapons  in  SMARTTS,  the  advanced  target  weapons  should  also 
be  included  to  enable  SMARTTS  to  provide  the  requisite  state-of-the-art 
tactical  encounters. 

dd.  (47)  The  AIT  should  be  expanded  to  include  additional  relevant 
capabilities.  Many  of  these  have  been  identified  in  the  initial 
specification  of  the  AIT,  but  not  implemented  on  the  preprototype. 
Additional  AIT  capabilities  include  the  conduct  of  a  cruise  missile  attack.. 
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avoidance  activities,  self-defense  attack  only  activities,  attack  decoys, 
and  so  on.  The  priority  of  AIT  capabilities  to  be  added  should  be 
determined  via  discussions  with  appropriate  naval  personnel,  with  a 
long-term  AIT  development  plan  put  into  place.  See  the  TAT  area  below, 
regarding  additional  discussions  on  the  AIT  (Section  5. 2. 2.1,  #51). 

ee.  (48)  The  capability  to  control  decoys  should  be  added  to  SMARTTS  and 
the  main  simulation  computer  group  for  use  during  ihe  real-time  attack 
center  exercises. 


5. 2. 1.4  Test  and  Evaluate  New  TAT  Concepts.  (49)  The  previous  subsection 
addressed  a  wide  range  <57  new  TaT  capabilities  for  investigation  and 
addition  as  a  part  of  SMARTTS  interface  to  the  21A41A  SCST.  A  cohesive 
long-term  investigative  program  should  be  developed  to  continually  evaluate 
new  TAT  concepts  implemented  in  tactics  training.  This  should  be  a 
permanent  program  structured  to  continually  upgrade  the  training  subsystem 
of  the  training  device.  In  particular,  this  program  should  receive  those 
TAT  concepts  that  have  been  found  via  laboratory  research  to  have  a  high 
potential  for  improving  the  effectiveness  of  training.  This  program  would 
then  implement  the  selected  concepts  on  a  prototype  basis,  and  evaluate 
these  potentially  effective  concepts  in  the  applied  training  setting,  during 
operational  training.  This  program  should  be  viewed  as  a  direct  follow-on 
to  the  SMARTTS  preprototype  test  and  evaluation,  recommended  above. 
Appropriate  data  collection  and  evaluation  capabilities  should  be  added  to 
SMARTTS  in  support  of  this  program.  Related  capabilities  to  support  this 
program  are  discussed  further  below  under  the  TAT  area.  Processing  Support 
(Section  5. 2. 2. 5,  #73)  and  Training  System  Management  (Section  5. 2. 2. 8,  #96). 


5. 2. 1.5  SCST  Site  Survey.  (50)  A  formal  survey  should  be  made  of  all  the 
SCST  sites  to  identify  training  assistance  technology  characteristics  that 
should  be  investigated  for  inclusion  as  a  part  of  SMARTTS,  or  other  training 
subsystems.  The  advent  of  SMARTTS  has  been  a  catalyst  to  the  generation  of 
training  assistance  technology  concepts  by  the  various  members  of  the  naval 
submarine  training  community.  The  site  personnel  have  been  exposed  to  the 
SMARTTS-type  of  technology  and  hence  have  generated  a  variety  of  ideas  in 
this  regard.  The  site  survey  should  be  the  means  for  collecting,  organizing 
and  identifying  relevant  TAT  concepts. 

It  is  essential  that  the  site  survey  be  more  than  simply  discussions 
with  the  various  site  personnel.  Rather,  a  carefully  prepared  survey 
instrument  should  be  developed  that  explores  in-deptn  the  range  of  training 
assistance  technology,  and  facilitates  the  drawing-out  of  new  concepts. 


5.2.2  TAT  RESEARCH  AND  DEVELOPMENT.  The  preprototype  SMARTTS  is  the 
initial  installation  of  training  assistance  technology  characteristics.  As 
such,  it  provides  those  feasible  character i sties  that  appear  to  have  the 
greatest  potential  for  improving  the  effectiveness  of  submarine  tactics 
training.  Additional  TAT  characteri sties  have  peen  identified  in  the 
earlier  analysis  work  leading  to  the  SMARTTS  preprototype,  although  not 
implemented  on  the  preprototype.  Furthermore,  tnose  major  TAT  capabilities 
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implemented  on  the  preprototype  (e.g.,  performance  indicators)  are  expected 
to  evolve  over  time  as  the  training  situation  and  training  technology 
evolves.  The  TAT  area  of  research  and  development,  as  herein  recommended, 
addresses  the  continual  development  of  training  assistance  technology  in  the 
generic  sense.  Whereas  the  previous  subsection  addressed  the  specific 
development  of  characteristics  pertinent  to  the  21A41A  SCST  and  work  to  be 
conducted  at  that  site,  the  recommendations  in  this  subsection  address  the 
broader  issues  of  TAT  capabilities.  Consequently,  the  recommendations  in 
this  subsection  are  typically  of  a  broader  nature,  and  not  necessarily  tied 
to  submarine  tactics  training.  Tnese  recommended  research  and  development 
issues,  which  are  generally  considered  to  require  laboratory  researcr., 
should  yield  specific  candidates  for  investigation  in  the  field  (e.g.,  as 
part  of  the  21A41A  site  research  program).  A  broad  range  of  research  and 
development  work  would  be  conducted  as  a  part  of  this  category  of  the 
research  and  development  plan,  continuing  the  thrust  to  develop  improved 
training  assistance  technology  for  enhancing  the  cost/effectiveness  of 
training.  Those  TAT  characteristics  deemed  ready  for  operational  training, 
or  operational  training  evaluation,  would  then  move  into  the  site  work 
category,  as  represented  by  the  above  subsection. 


5.2.2. 1  Automatic  Interactive  Target.  (51)  The  automatic  interactive 
target  (AIT)  is  a  relatively  new  simulation  concept.  It  has  the  potential 
of  being  the  primary  means  for  dissemination  of  intelligence  information 
regarding  target  actions  to  the  fleet.  Furthermore,  it  can  improve  the 
effectiveness  of  tactical  training  by  off-loading  the  instructor  and  thus 
providing  him  with  more  time  for  other  aspects  of  the  training  process,  and 
providing  more  realistic  and  representative  target  actions  in  response  to 
ownship  and  the  tactical  situation.  Finally,  the  AIT  can  provide  a  means 
for  close  analysis  of  the  distribution  of  target  actions  on  the  basis  cf 
ownship  tactics  and  other  situational  factors.  A  range  of  AIT  capabilities 
have  been  identified  as  a  part  of  an  earlier  analysis  effort  prior  to 

development  of  the  preprototype 's  AIT  model.  The  preprototype  AIT  is  a 
single  target  submarine  with  system  capabilities  and  a  decisionmaKing 
structure  modeled  after  a  particular  enemy  target.  This  AIT  model 

represents  the  initial  step  in  the  development  of  comprehensive  AIT 
capabilities,  pertaining  to  multiple  targets,  and  so  on. 

It  is  recommended  that  a  long-term  research  and  development  effort  be 

initiated  to  evolve  AIT  models  as  an  integral  component  of  the  training 
system.  This  long-term  effort  should  include  the  following  major  parts: 

a.  (52)  Identification  and  prioritization  of  AIT  capabilities.  Tnis 
should  include  a  time  schedule  plan  for  the  development  of  AIT 

capabilities.  Potential  capabilities  to  be  planned  for  should  include: 

1.  Additional  submarine  targets 

2.  Multiple  AITs  in  a  single  scenario. 

3.  AIT  consort  operations 

4.  Surface  and  air  AITs 

5.  Battle  group,  convoy,  etc.,  AITs 
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b.  (53)  The  AIT  models  should  be  developed  in  accordance  with  the 
priority  established  above.  This  includes  the  collection  of  necessary 
intelligence  information,  the  design  or  modification  of  decision  structures, 
the  development  of  algorithms,  and  testing  of  the  model. 

c.  (54)  The  AIT  models  should  be  validated  through  empirical 
investigation  in  the  applied  training  setting.  Tnese  evaluations  should  not 
be  aimed  at  identifying  the  acceptability  or  unacceptability  of  the  AIT,  but 
rather  should  be  aimed  at  identifying  the  valio  proportions  and  invalid 
proportions  of  the  model,  with  an  appropriate  feedback  loop  to  the  AIT 
development  process  to  assure  modification  and  improvement  of  the  models  as 
necessary.  The  validation  should  be  based  on  botn  intelligence  information, 
and  interaction  with  experienced  submariners. 

d.  (55)  The  AIT/instructor  interface  should  be  closely  investigated  to 
provide  effective  monitoring  and  control,  as  necessary.  This  should  include 
the  identification  of  instructor  information  requirements  and  control 
requirements;  the  research  and  development  of  display  concepts  for  providing 
necessary  information;  and  the  research  and  development  of  control  concepts 
to  enable  the  instructor  to  manipulate  the  AIT  in  various  ways  to  achieve 
desired  training  objectives.  For  example,  it  may  be  desirable  to  provide  a 
decision-tree  diagram  on  a  display  format  while  the  AIT  is  running  so  as  to 
enable  the  instructor  to  keep  track  of  the  AIT's  decision  process.  This 
display  format  may  also  be  effective  in  classroom  instruction  regarding 
enemy  target  characteristics  and  actions.  For  example,  the  instructor  may 
wish  to  dissect  the  AIT  in  the  classroom  to  focus  on  subsets  of  the  AIT's 
decision  process  and  likely  actions  (e.g.,  demonstration  of  various  baffle 
clearing  maneuvers  in  a  classroom  context,  with  probabi 1 ities  of  different 
course  changes  on  each  leg  of  a  particular  baffle  clearing  maneuver). 

e.  (56)  A  procedure  should  be  developed  for  the  timely  collection  of 
recent  intelligence  information,  and  its  transposition  into  an  updated  AIT 
model.  This  would  have  to  be  done  on  a  regular  basis  between  naval 
intelligence  and  training  activities  to  transfuse  the  most  recent 
intelligence  information  into  the  models,  and  transmit  that  to  the 
respective  sites.  Regular  AIT  update  loads  should  be  provided  to  each  site 
on  a  periodic  basis.  This  would  be  a  major  part  of  the  continual 
improvement  of  the  AIT. 


5 .2.2.2  Alternative  Actions.  (57)  Learning  by  example  is  an  important 
construct  of  thi  instructional  process.  The  alternative  tactics 
characteristics  of  SMARTTS  provide  tools  to  tr.e  instructor  to  rapidly 
generate  alternative  examples  and  their  associatea  performance-related 
information,  and  to  provide  comparative  summaries  of  those  alternatives. 
Alternative  actions,  as  used  herein,  is  tr.e  same  as  alternative  tactics 
except  that  it  is  applicable  to  any  training  application,  not  just  tactics. 
Alternative  tactics  should  be  viewed,  therefore,  as  a  subset  of  alternative 
actions.  Alternative  actions  is  the  generic  terminology  representing  the 
concept  of  using  the  training  device  character islics  to  generate  multiple 
alternative  sets  of  actions  to  assist  in  learning  oy  example. 
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The  alternative  actions/alternative  tactics  capability  on  SMARTTS 
requires  the  entry  of  substantial  setup  parameters  to  generate  the  detailed 
information  relevant  in  the  complex  tactical  situation.  It  is  recommended 
that  a  research  program  be  undertaken  to  improve  the  instructor  interface 
with  alternative  actions-type  capabilities.  This  would  require  the 
identification  of  various  ways  of  implementing  alternative  actions,  and 
their  evaluation  from  a  human-interface  and  training  standpoint.  It  is 
expected  that  substantial  trade-offs  do  exist  between  the  human  interface 
considerations  and  the  generation  of  information  necessary  from  a  training 
standpoint.  The  impact  of  these  trade-offs  are  also  likely  to  vary  as  a 
function  of  other  aspects  of  the  training  situation.  Since  the  alternative 
actions-type  capability  has  such  substantial  potential  for  improving  the 
effectiveness  of  training,  it  warrants  the  attention  of  an  in-depth  research 
program  to  achieve  effective  interface  designs  that  will  insure  its  use 
during  the  training  process. 

Several  considerations  for  the  alternative  actions/instructor  interface 

are: 


a.  Automatic  selection  of  alternative  actions  for  ownship  and  targets 
via  a  computer-based  model  of  the  tactical  situation.  This  model  would  key 
on  particular  parameters  in  the  situation  as  they  develop  to  automatically 
recommend  several  ownship  and/or  target  alternative  tactical  actions  for 
investigation  by  the  instructor  and  trainees;  this  model  could  also  provide 
the  major  points  of  rationale  for  selection  of  the  alternatives.  Finally, 
the  model  would  bring  up  the  alternatives  ^or  investigation  as  the  result  of 
a  single  input  command  by  the  instructor. 

b.  Available  alternative  actions  could  be  categorized  for  different 
segments  of  a  tactical  encounter.  For  example  several  fundamental  TMA 
approaches  are  available  (i.e.,  point-lead;  lead-lag;  point-lag).  Within 
each  of  these  alternative  categories  various  subcategories  are  available. 
For  example,  in  the  lead-lag  TMA  approach  tactic  various  ownship  course 
changes  and  the  times  at  which  those  changes  are  initiated  are  available.  A 
repertoire  of  these  alternatives  relative  to  the  actual  actions  chosen 
during  the  real-time  exercise  could  be  rapidly  generated  by  the  computer  and 
presented  to  the  instructor  for  selection.  During  the  alternative  tactics 
session,  therefore,  the  instructor  would  select  a  set  of  alternative  actions 
for  investigation  from  the  complete  set  generated  and  made  available  by  the 
computer. 

c.  Alternative  tactical  actions  could  be  predetermined  for  eacn 
exercise  prior  to  its  use,  scripted  (i.e.,  canned),  and  stored  along  with 
the  basic  exercise  parameters.  The  instructor  could  then  select  alternative 
actions  from  the  this  predetermined  set  available  with  each  exercise. 

d.  Various  control/display  characteristics  are  available  for  the 
instructor  interface.  SMARTTS  currently  requires  the  instructor  to  punch  in 
the  necessary  ownship  and  target  maneuver  parameters  for  each  leg  of  the 
alternative  tactics  session.  It  is  expected  that  a  more  efficient 
human-engineered  interface  is  possible  tnrough  tne  use  of  other  hardware 
and/or  software  presentation  characteristics.  Examples  include:  a 
joystick,  mouse,  or  ball-and-trac*  for  positioning  a  cursor  to  the  next 
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incremental  position  of  ownship,  allowing  the  computer  to  calculate  tne 
actual  maneuever  parameters;  analog  type  of  display  presentation  and  control 
capabilities  on  the  plasma  entry  device  for  selection  of  ownship  or  target 
actions. 


5. 2. 2. 3  Performance  Indicators.  (58)  The  performance  indicators  are  key 
characteristics  of  training  assistance  technology,  and  the  SMARTTS 
preprototype.  Together  with  the  tactical  variaoles  the  performance 
indicators  form  the  bulk  of  information  that  is  provided  to  the  trainees  and 
the  instructor  via  the  information  displays.  A  set  of  objective  and 

subjective  performance  indicators  have  been  developed  for  the  SMARTTS 
preprototype,  addressing  basic  submarine  tactics  training.  This  set  is  by 
no  means  comprehensive  for  basic  submarine  tactics  training,  much  less  for 
the  entire  scope  of  submarine  tactics  training.  Rather,  it  is  a 
representative  set  of  P Is.  A  long-term  research  and  development  program 
should  be  initiated  to  investigate  and  develop  Pis  for  tactics  training.  It 
is  expected  that  the  Pis  will  evolve  over  time  as  the  tactical  situation  and 
training  technology  change,  hence  the  need  for  continual  development  of 
Pis.  Several  issues  are  relevant  to  the  research  and  development  of  Pis: 

a.  (59)  Representative  Pis  should  be  validated  with  regard  to  their 
relevancy  to  the  training  process. 

b.  (60)  The  appropriateness  of  minimum  performance  standards  with 

regard  to  the  achievement  of  terminal  training  objectives  should  be 

investigated.  These  performance  standards  would  relate  to  a  particular 
level  of  performance  with  regard  to  one  or  more  Pis. 

c.  (61)  Both  objective  and  subjective  Pis  need  to  be  continually 

developed  to  keep  pace  with  the  ever  changing  tactical  situations  and 
needs.  Information  and  feedback  display  formats  should  be  developed 

together  with  the  Pis  so  as  to  best  convey  the  desired  information.  The 
instructor  console  interface  character-! sties  should  also  be  developed  in 
conjunction  with  the  subjective  Pis  to  facilitate  their  entry  by  the 
instructor. 

d.  (62)  Individual  Pis  and  team-level  Pis  should  be  developed.  The 

team-level  Pis  should  represent  summary  performance  by  the  complete  team  or 
subteam  elements.  They  should  address  not  only  the  tactical  outcome  of 
particular  segments  of  the  exercise,  but  also  relevant  team-related 

characteristics. 

e.  (63)  Summary  Pis  should  be  developed  for  the  purpose  of  evaluating 

team  performance  with  regard  to  meeting  terminal  training  objectives.  The 
Pis  developed  in  the  preprototype  SMARTTS  were  designed  to  provide 

training-related  information.  This  should  oe  tne  purpose  of  the  majority  of 
Pis.  In  addition,  however,  the  need  does  exist  for  Pis  with  wnich  to 

evaluate  training  progress,  relative  team  performance,  and  tactical 

readiness.  The  TEF  metric  developed  by  SUBlAMT  is  an  example  of  a  summary 
PI.  This  type  of  information  is  necessary  for  the  various  research  and 

development  efforts  (i.e.,  to  evaluate  tne  ef fecti veness  of  various  TAT 
capaoi 1 i ties) . 
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f.  (64)  An  analysis  should  be  undertaken  to  identify  performance 
criteria  from  all  phases  of  tactics  training,  for  which  the  development  of 
appropriate  Pis  can  be  undertaken. 

This  research  and  development  program  should  seek  to:  (a)  develop 
applied  performance  indicators  for  use  in  training,  and  (b)  develop  manual 
tools  for  use  by  the  instructor  and  automated  tools  to  collect  the  necessary 
information  and  generate  the  appropriate  indicators.  The  submarine  tactical 
training  situation  is  extremely  complex  and  does  not  readily  lend  itself  to 
the  assessment  of  performance,  particularly  at  the  team-level.  As  a  result, 
the  more  encompassing  summary  performance  indicators  tend  to  be  subjective. 
The  research  and  development  program  should,  therefore,  also  seek  to  develop 
tools  for  use  by  instructors  in  the  subjective  evaluation  process. 

A  survey  should  be  conducted  of  the  Fleet,  SCST  sites,  and  the  PXO/PCO 
training  program  to  identify  performance  criteria  leading  to  the  development 
of  performance  indicators.  An  SCST  site  survey  was  recommended  above  with 
regard  to  the  21A41A  site  work  (Section  5. 2. 1.5,  #50).  It  may  be 
appropriate  to  expand  that  survey  to  address  candidate  criteria  and 
performance  indicators,  and  also  expand  it  to  the  Fleet  and  PXO/PCO  training 
programs. 


5. 2. 2. 4  Information  Presentation.  (65)  The  SMARTTS  information  displays 
are  the  primary  interface  with  the  trainees  and  the  instructor. 
Characteristics  of  the  information  display  interface  should  have  a 
tremendous  impact  on  the  effectiveness  with  which  information  is 
transmitted,  and  hence  of  the  training  process.  The  design  of  information 
displays  today  is  largely  an  art  form,  and  often  relatively  ineffective.  Of 
the  research  that  has  been  conducted,  almost  none  has  addressed  display 
design  to  achieve  an  effective  information  transmission  medium  with  regarc 

to  the  training  process.  A  wide  range  of  issues,  therefore,  require 

investigation.  These  include: 

a.  (66)  The  training  effectiveness  of  fundamentally  different  display 

dimensions  (e.g.,  alphanumeric  information  versus  graphic  information)  with 
regard  to  the  training  of  specific  skills  and  knowledge. 

b.  (67)  The  use  of  color. 

c.  (68)  Unique  display  requirements  pertaining  to  the  trainee;  unique 
display  requirements  pertaining  to  the  instructor. 

d.  (69)  The  use  of  aural  information  either  alone  or  in  combination 

with  visual  information.  The  aural  information  dimension  has  been  seldom 
used  as  a  training  aid,  in  comparison  with  the  visual  dimension.  Several 
unique  aspects  are  related  to  the  aural  dimension,  however.  The  most 
obvious  is  that  it  presents  additional  information  dimensions  that  can  be 
used  in  parallel  with  visual  information.  For  example,  wnile  the  instructor 
is  attending  to  the  particular  aspects  of  a  problem  away  from  the 
instructor's  console  he  can  be  receiving  status  information  and  cues 
aurally.  Although  the  instructor  does  typically  receive  aural  information 
each  day,  little  attention  has  been  given  to  the  various  aspects  of 
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presenting  such  information  (e.g.,  timing,  quantity,  format,  etc.)*  In 
those  training  situations  where  the  instructor  must  interact  heavily  witn 
one  or  more  students,  away  from  immediate  access  to  visual  information  on  an 
instructor's  console,  aural  information  provided  to  the  instructor  may  be 
the  most  effective  means  of  giving  him  cues  and  status  information.  Also, 
aural  information  may  be  an  effective  means  of  providing  guidance  and 
immediate  feedback  to  trainees  during  a  particular  training  exercise.  For 
example,  aural  feedback  or  guidance  information  may  be  presented  to  a  single 
individual  without  affecting  other  trainees  in  the  team.  With  the  existing 
capability  of  providing  relatively  good  computer-generated  voice  synthesis 
at  low-cost,  aural  information  may  be  a  means  of  providing  automated 
assistance.  Research  should  be  conducted  to  determine  the  capabilities  and 
limitations  of  aural  information  with  regard  to  the  various  aspects  of  the 
training  process.  This  research  should  address  both  the  instructor  (i.e., 
from  a  status  and  cue  standpoint)  and  the  student  (i.e.,  from  the  standpoint 
of  providing  guidance  during  the  training  exercise,  as  well  as  for  providing 
additional  feedback  information  during  the  debriefing  session  —  for 
example,  while  the  entire  team  is  receiving  visual  information  during  the 
debrief  individual  operators  may  be  receiving  additional  aural  information 
tailored  to  their  specific  problem  aspects).  The  research  should  address 
both  manual  and  computer-generated  automated  aural  information. 

e.  (70)  The  use  of  three-dimensional  display  representations  for  the 
three  dimensional  submarine  environment.  An  aspect  of  substantial  potential 
impact  on  training  effectiveness  is  the  use  of  three-dimensional  display 
presentation  techniques  (e.g.,  object  rotation;  transparent  objects)  on  a 
two-dimensional  CRT  display.  Several  aspects  of  the  submarine  tactical 
problem  are  heavily  dependent  on  the  three-dimensional  environment,  although 
they  have  traditional ly  been  viewed  as  a  two-dimensional  problem  due  to 
equipment  and  conceptual  limitations.  Unfortunately,  too  often  the 
operators  themselves  view  the  problem  as  two-dimensional  due  to  display 
limitations;  this  must  certainly  have  an  adverse  impact  on  operational 
effectiveness.  Figures  16  and  17  address  the  target/firing  point 
relationship  and  the  weapon/target  relationship,  providing  examples  of  a 
possible  three-dimensional  display  format.  During  an  approach  and  attack  on 
a  target,  ownship  seeks  to  maneuver  to  within  weapon  range,  and  at  the 
greatest  possible  range  consistent  with  his  objectives  launch  the  weapon. 
The  weapon  launch  range  is  dependent  upon  the  weapon's  characteristics 
(e.g.,  speed,  amount  of  fuel),  target  angle  on  the  bow,  environmental 
characteristics  (e.g.,  existence  or  absence  of  a  layer),  and  other  factors. 
The  probability  of  kill  would  depend  upon  the  particular  combination  of 
factors  at  the  time  the  weapon  was  launched.  If  a  particular  probability  of 
kill  was  wanted  (e.g.,  60  percent)  then  a  three-dimensional  surface  could  be 
located  around  the  target  representing  a  complete  set  of  points  at  which  a 
60  percent  probability  of  kill  could  be  acnieved.  This  surface  would  be 
irregular  and  dependent  upon  the  variety  of  ownship,  weapon,  target  and 
environmental  characteristics  in  the  particular  target  situation.  The 
SMARTTS  preprototype  provides  a  firing  point  wnich  is  a  single  point  in 
space  representing  tne  desired  culmination  of  the  approach,  and  the  point  at 
which  the  weapon  would  be  fired  from  ownship.  This  single  point,  although 
useful  in  the  training  context,  is  incomplete  in  that  it  implies  that  a 
single  point  in  space  is  the  desired  firing  position,  whereas  many  firing 
points  actually  exist.  The  single  firing  point  in  tne  SMARTTS  preprototype 
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would  be  a  single  point  on  the  surface  of  the  three-dimensional  firing  point 
envelope  surrounding  the  target.  Figure  16  provides  an  illustration  of  wnat 
this  type  of  envelope  might  look  like.  A  display  format  of  this  type  would 
appear  to  be  useful  in  addressing  approach  tactics,  weapon  launch  point, 
target  characteristics  impacting  ownship  tactics  and  weapon  launch,  and  so 
on. 

Weapon  deployment  is  another  three-dimensional  problem  that  is 
relatively  difficult  tc  address  in  the  current  training  context.  Figure  17 
illustrates  a  three-dimensional  diagram  of  the  torpedo  and  its  acquisition 
cone.  This  type  of  diagram  might  be  useful  for  illustrating  the 
three-dimensional  search  and  acquire  characteristics  of  the  torpedo  in  a 
manner  similar  to  that  discussed  above  with  regard  to  the  firing  point. 

f.  (71)  The  use  of  motion  on  instructional  displays.  Motion  has  seldom 
been  used  on  a  display  surface  in  naval  tactics  or  training.  Motion  is  an 
information  dimension  that  has  unique  aspects  with  regard  to  human 
preception.  It  may  facilitate,  for  example,  the  learning  of  skills  to 
project  an  evolving  situation  out  in  time.  For  example,  if  coupled  with  the 
three-dimensional  display  suggested  above  regarding  weapon  deployment 
(Figure  17),  the  capability  to  have  the  torpedo  go  through  three-dimensional 
search  patterns  in  fast-time  via  presentation  on  a  display  may  greatly  aid 
the  student  in  understanding  the  various  search  patterns  of  the  weapon,  and 
their  capabilities  and  limitations  with  regard  to  various  target  types  and 
target  actions.  Motion  is  used  today  on  several  SMARTTS  displays  as  the 
situation  evolves  in  real-time  or  fast-time.  However,  the  motion  on  these 
displays  is  purely  an  artifact  of  their  update  rate  to  provide  appropriate 
status  information.  Nevertheless,  the  motion  of  ownship  and  targets  when 
run  in  a  fast-time  configuration  does  appear  to  provide  a  more  effective 
mode  of  information  presentation  regarding  the  interaction  between  platform 
in  the  particular  situation,  as  compared  with  showing  the  completed  history 
tracks  of  ownship  and  the  targets  at  completion  of  the  exercise.  It  is 
expected  that  if  motion  were  directly  used  to  illustrate  aspects  of  the 
problem,  such  as  the  torpedo  search  activities,  the  effectiveness  of  the 
training  in  these  instances  would  substantially  improve. 


g.  (72)  Plasma-type  touch  entry  instructor  command  sequences.  The 
plasma  entry  device  is  a  unique  medium  in  that  it  provides  information 
presentation,  as  well  as  accepts  input  commands.  The  discussions  herein 
regarding  the  plasma  entry  device  would  pertain  equally  to  other  display 
surfaces  that  are  touch  sensitive.  The  plasma  capability  effected  on  the 
SMARTTS  preprototype  is  considered  to  be  the  base  level  of  plasma-relateo 
capability.  English  language  text  is  used  in  place  of  acronym  command 
sequence;  the  number  of  response  options  available  to  the  instructor  is 
controlled  and  related  to  each  particular  situation;  information  relevant  to 
the  particular  input  options  available  is  controlled  and  presented;  and  tne 
entire  input  sequence  is  logically  structured  and  related  to  tne  particular 
subset  tnat  the  instructor  is  dealing  with  at  any  time.  Tnese  capabilities 
certainly  improve  the  ease  of  interacting  with  the  training  device,  as  well 
as  substantially  improve  the  effectiveness  of  learning  now  to  use  tne 
training  device.  The  instructor's  console  becomes  a  training  aid  in  itself 
due  to  the  inherent  capabilities  of  the  plasma  display/entry  device.  Two 
additional  levels  of  increasing  sophistication  centered  around  the  plasma 
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device  have  been  previously  suggested:  (a)  configuration  of  the  information 
presentation  formats  and  command  entry  actions  tailored  to  instructor 
characteristics;  and  (  b)  use  of  artificial  intelligence-related  capabilities 
to  improve  the  functioning  of  the  instructor's  console  via  the  plasma-type 
interface.  For  example,  the  information  presentation  and  command  entry 
characteristics  may  be  improved  through  the  use  of  better  display  formats 
(e.g.,  a  much  greater  use  of  graphics  for  information  presentation  and  for 
command  entry).  The  sequences  of  system  operation  may  be  improved  as  a 
result  of  a  better  understanding  of  the  instructor  tasks  in  using  the 
system.  Improvement  in  the  instructor  command  sequence  could  be 
accomplished  automatically  by  the  machine  itself.  The  machine  can  monitor 
the  usage  patterns  by  the  instructor,  and  devise  an  optimum  usage  pattern 
minimizing  the  number  of  command  steps.  Research  should  be  conducted,  in 
conjunction  with  that  recommended  above  pertaining  to  display  formats,  etc., 
to  devise  the  principals  of  effective  formatting  and  command  sequence 
structures  for  the  instructor's  console.  These  should  be  centered  around 
the  touch-sensitive  command  entry  surface. 

The  plasma-type  touch  entry  surface  facilitates  operation  since  it 
provides  information  and  accepts  commands  in  a  compatible  format  with  the 
human  instructor.  The  plasma-type  entry  device  does  not  directly  impact  the 
types  of  processing  functions  that  occur  behind  the  interface,  with  the 
exception  that  it  does  enable  the  use  of  interface  functions  that  are  more 
amenable  to  the  instructor  operator  since  it  is  less  constrained  by  the 
engineering  mechanics  of  the  interface.  That  is,  when  entering  information 
via  an  alphanumeric  keyboard  the  instructor  is  adapting  to  the  rigid 
characteristics  of  the  entry  medium,  in  the  form  of  learning  appropriate 
acronyms,  etc.  «ith  the  plasma  entry  device,  the  instructor  simply  reads  in 
English-like  text  and  points  to  the  selections.  This  can  be  further 
extended  as  noted  immediately  above  by  the  use  of  graphical  information  and 
selection  options,  in  further  alignment  with  the  human  capabilities  of  the 
instructor  operator  (i.e.,  he  typically  can  read  English  language  text  quite 
well,  as  well  as  point  to  things;  also,  presumably,  visual  graphical 
information  is  a  more  effective  means  of  presentation  and  selection  in 
certain  instances).  This  facilitation  of  the  direct  human  interface  is 
likely  to  further  enhance  the  interface  with  the  computer  system  processing 
and  the  use  of  artificial  intel 1 igence-related  processing  functions,  since 
it  fosters  a  more  symbiotic  relationship  between  the  instructor  and  the 
device.  The  characteristics  that  can  be  provided  at  this  highest  level  of 
interface  functioning  are  not  readily  known  at  this  time.  A  training  device 
manufacturer  would  be  unlikely  to  provide  many  characteristics  in  this 
regard  today.  However,  this  should  be  the  direction  for  research  since  it 
has  a  tremendous  potential  for  improving  the  relationship  between  the 
instructor  and  the  training  device,  and  greatly  enhancing  the  training 
process. 


5. 2. 2. 5  Processing  Support.  (73)  Major  unaerlying  characteristics  of 
SMARTTS  center  on  its  capabilities  to  process  data,  generating  information 
such  as  the  performance  indicators,  alternative  tactics,  and  so  on.  Other 
information  processing  characteristics  may  be  relevant  in  support  of  the 
training  process.  These  include  real-time  processing  characteri sties  to 
support  the  exercise  development,  conduct  of  training,  and  briefing 
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sessions,  as  well  as  the  longitudinal  analysis  capabilities  identified  in 
the  SMARTTS  Type  A  specification.  Research  should  be  conducted  to 
investigate  the  effectiveness  of  various  support  processes  to  enhance  the 
conduct  of  training  and  to  support  the  site  management  functions.  Relevant 
issues  include: 

a.  (74)  An  investigation  should  be  initiated  to  identify  potentially 
useful  support  processing  characteristics.  These  would  include 
characteri sties  to  assist  the  instructor  in  monitoring  and  evaluating 
relevant  exercise-related  aspects  during  the  real-time  problem  execution;  in 
designing  and  evaluating  exercise  scenarios;  in  evaluating  team  and 
individual  performance  after  completion  of  the  exercise;  and  in  generating 
and  providing  relevant  feedback  information  to  the  students  following 
completion  of  the  exercise.  The  investigation  should  also  address  long-term 
analysis  capabilities  to  support  necessary  longitudinal  analysis  functions 
(e.g.,  evaluation  of  training  system  effectiveness;  tactical  team  readiness). 

b.  (75)  The  development  of  statistical  analysis  routines  to  combine  and 
evaluate  training  performance-related  data  across  multiple  exercises.  These 
routines  should  permit,  for  example,  the  investigation  of  particular 
tactical  training  and  operational  problems  (e.g.,  maneuvering  to  a  firing 
position)  related  to  particular  types  of  exercises  (e.g.,  barrier  patrol); 
the  tactical  performance  of  various  teams  across  a  variety  of  different 
types  of  tactical  engagements;  the  training  effectiveness  improvement  of 
teams  as  a  function  of  the  amount  and/or  type  of  training  received;  and  the 
identif ication  of  individual  team  proficiencies  and  deficiencies  on  which  to 
focus  subsequent  training  emphasis.  Specifications  for  the  analysis 
routines  would  be  identified  in  the  above  recommended  investigation,  with 
the  actual  routines  developed  under  this  particular  task  issue. 

c.  (76)  The  development  of  processing  routines  to  support  tne 
instructor  in  assessing  the  input  characteristics  of  particular  combat 
teams.  The  input  characteristics  would  be  based  on  the  prior  performance  of 
the  respective  teams  in  selected  training  exercises.  These  characteristics 
could  be  used  in  whole,  or  in  part,  to  diagnostically  evaluate  a  combat 
team's  capabilities  and  limitations,  and  substantially  aid  the  instructor  in 
tailoring  a  training  program  to  meet  their  specific  needs. 

d.  (77)  A  standardized  scenario  structure  should  be  developed  for 
categorization  of  scenarios  and  their  parts.  This  would  enable  tne 
establishment  of  a  data  base  across  multiple  scenarios  and/or  scenario  parts 
for  evaluation  of  training  performance,  and  for  the  selection  of  appropriate 
exercises  to  meet  particular  training  needs.  Each  training  scenario,  today, 
is  considered  as  unique.  However,  it  is  suggested  that  only  several 
basically  different  scenarios  or  scenario  parts,  in  fact,  exist  in  specific 
training  applications  (e.g.,  submarine  tactics);  many  scenarios  or  scenario 
parts  are  suggested  as  being  similar,  perhaps  differing  only  in  minor 
aspects.  The  SMARTTS  control  group  data  collection  effort  developed  a 
series  of  performance  evaluation  tools  to  assess  different  aspects  of 
tactical  team  performance.  These  tools  included  categorization  of  different 
scenario  parts,  so  as  to  enable  comparison  of  performance  across  aifferent 
scenarios.  This  may  be  a  good  starting  point  for  development  of  the 
standardized  scenario  structure.  Processing  capabilities  should  be 
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developed  to  enable  the  instructor  to  index  tnrougn  scenarios  identifying 
their  relevant  parts,  selecting  scenario  parts  to  configure  a  complete 
exercise,  and  selecting  scenarios  to  achieve  particular  training  objectives. 

e.  (78)  Processing  characteristics  should  be  developed  to  assist  the 
instructor  in  the  setting  up  and  managing  of  the  instructional  process 
across  multiple  courses  and  student  teams.  In  particular,  these  would  be 
computer-managed  instruction  (CMI)  capabilities. 

f.  (79)  Processing  characteristics,  in  addition  to  those  currently  on 
SMARTTS,  should  be  developed  for  use  by  the  instructor  in  actually 
conducting  the  training  process.  These  may  include  real-time  analysis 
routines  to  investigate  different  aspects  of  a  particular  exercise,  to 
present  multiple  performance  indicators  and/or  tactical  variables,  and  to 
investigate  a  variety  of  alternative  actions.  The  discussion/recommendation 
above  concerning  alternative  tactics  (Section  5. 2. 2. 2,  #57)  would  be 
relevant  in  this  regard,  as  well  as  the  development  of  new  performance 
indicators  (Section  5. 2. 2. 3).  The  intention  of  this  issue  is  to  develop 
additional  processing  characteristics  to  analyze  and  present  the 
performance-rel ated  information  concerned  with  the  prescenario  and 
postscenario  exercise  briefings.  It  might  include  an  on-the-spot  analysis 
of  a  particular  exercise  with  regard  to  overall  performance  indicators, 
comparison  of  performance  in  a  particular  scenario  with  the  performance  of 
other  combat  teams,  and  comparison  of  the  performance  of  a  particular  combat 
team  with  their  previous  performance. 

Minimal  support  processing  characteristics  have  been  included  in  the 
preprototype  SMARTTS.  These  have  been  recognized  in  the  type  A  SMARTTS 
specification  as  necessary  for  the  conduct  of  effective  training,  both  from 
the  standpoint  of  the  training  process  itself  as  well  as  from  the  larger 
management  aspects.  In  many  respects  the  development  of  processing  support 
capabilities  is  not  a  research  issue,  but  rather  a  developmental  issue. 


5. 2. 2. 6  Instructor-Specific  Characteristics.  (80)  Most  of  the 
characteristics  included  in  the  preprototype  SMARTTS,  and  addressed  in  this 
research  and  development  plan  are  intended  to  aid  the  instructor  in 
conducting  his  functions.  Many  of  these  also  pertain  to  the  trainee  in  that 
tney  enable  the  instructor  to  generate  guidance  and/or  feedback  information 
in  more  effective  formats  for  presentation  to  the  trainees.  Other 
characteristics  are  specifically  directed  at  assisting  the  instructor  in 
conducting  his  functions,  and  do  not  directly  interface  with  the  trainees. 
Those  are  included  in  tnis  category.  Several  characteristics,  in  addition 
to  those  identified  earlier,  are  recommenced  to  be  investigated.  These 
include  the  following: 

a.  (81)  The  development  of  guidelines  for  instructor  console  design. 
It  may  oe  possible  to  develop  an  effective  general  pjrpose  instructor's 
conso fe  to  cover  a  large  portion  of  training  devices.  Tnis  appears  feasible 
due  to  the  general  purpose  nature  of  many  instructor/program  operator 
consoles,  which  differ  between  devices  only  in  tne  particular  instructor 
interface  information.  A  standardized  general  purpose  instructor's  console 
would  have  many  advantages,  in  addition  to  cost  benefits.  Tne  necessary 
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training  of  instructors,  for  example,  would  be  minimized  if  they  have  had 
previous  experience  as  an  instructor,  or  had  earlier  experience  in  one  form 
or  another  with  an  instructor's  console.  Considerable  research  effort  could 
be  placed  into  designing  a  general  purpose  instructor's  console,  the 
benefits  of  which  would  therefore  translate  into  multiple  training 
applications  (i.e.,  the  cost  of  research  and  development  would  be  amortized 
across  multiple  training  devices).  This  may  become  a  reality  due  to  the 
increasingly  generalized  nature  of  operational  consoles  in  many  areas  of 
naval  operations.  It  behooves  the  training  community  to  lead  the  way  in  the 
design  of  a  general  purpose  instructor's  console,  ratner  than  have  it  forced 
upon  them  as  a  result  of  evolving  operational  systems.  Hence,  the  need  to 
initiate  an  instructor's  console  investigation  effort. 

b.  (82)  An  instructor  interface  language  should  be  developed  to 
facilitate  the  instructor's  interaction  with  the  training  device,  and 
greatly  assist  his  using  that  device  to  perform  his  various  functions  (i.e., 
dealing  with  exercise  design,  control  and  monitoring  of  training,  trainee 
briefing  and  so  on).  The  typical  instructor  in  military  training  systems 
has  a  good  operational  background  (e.g.,  submarine  operating  experience  for 
a  submarine  tactical  instructor),  although  a  limited  instructional  and 
computer  background,  particularly  with  regard  to  operating  specific  training 
devices.  It  is  desirable  to  have  a  computer  language  developed  to 
facilitate  the  instructor's  interface  with  the  training  device.  This 
interface  language  should  permit  the  instructor  to  interact  using  somewhat 
standardized  operational  terminology,  instructional  terminology,  and/or 
near-English  terminology.  It  should  be  designed  to  accommodate  those 
activities  normally  performed  by  the  instructor  when  interfacing  with  the 
training  device.  With  regard  to  exercise  development,  for  example,  this 
language  should  accept  descriptive  commands  by  tne  instructor  in  the  form  of 
the  normal  operational  parameters  (e.g.,  whereas  the  computer  manipulates 
targets  on  an  X,  Y,  Z  grid,  operational  personnel  often  view  the  geographic 
situation  in  terms  of  range,  bearing,  and  depth/elevation).  Furthermore, 
the  interface  language  should  be  structured  to  contain  a  large  set  of  macro 
functions  that  readily  translate  the  instructor’s  needs  into  exercise, 
performance  indicator,  cue,  feedback  display,  etc.,  modifications.  For 
example,  the  capability  should  be  provided  to  enable  the  instructor  to 
readily  tailor  feedback  displays  to  particular  exercises;  that  is,  tne 
instructor  should  have  the  capability  to  readily  modify  a  particular  display 
format  to  fit  the  needs  of  a  particular  exercise.  This  macro  function 
should  enable  the  instructor  to  rapidly  provide  the  information  necessary  to 
configure  the  display  in  terms  as  close  as  possiole  to  those  operationally 
used,  and  then  have  the  computer  automatically  configure  the  display.  This 
might  be  accomplished  using  a  digital  interface  drawing  tablet,  with 
appropriate  parameter  data  readily  accessible,  and  with  standardizea 
manipulation  algorithms  available.  The  instructor  interface  language  would 
have  the  advantage  of  requiring  minimal  training  on  the  part  of  the 
instructor  to  enable  maximal  use  of  the  capabilities  resident  in  tne 
training  device.  The  obvious  advantage  is  more  time  available  for  the 
conduct  of  necessary  training  functions,  and  greater  flexibility  and 
capability  to  conduct  those  functions.  It  may,  or  may  not,  be  feasible  to 
have  a  standardized  general  purpose  instructor's  language,  applicable  across 
multiple  platform  types  (e.g.,  air,  surface,  subsurface).  Most  instructor 
functions  are  generically  similar  across  platform  types,  although  the 
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specific  data  they  deal  with,  the  training  methodologies  used,  and  the 
device  capabilities  do  tend  to  differ.  Hence,  it  may  be  possible  to  have  a 
generic  instructor's  language  of  which  the  bulk  of  processing  capability  is 
common  across  all  training  devices,  but  in  which  the  specific  interface 
language  and  perhaps  some  of  the  command  characteristics  need  to  be  tailored 
to  the  particular  training  application. 

c.  (83)  An  exercise  development  simulation  capability  should  be 
developed,  and  implemented  on  most  training  devices.  A  flexible  high  speed 
simulation  capability  is  necessary  to  enable  the  instructor  to  review 
existing  exercises,  modify  an  exercise,  or  create  a  new  exercise.  Review, 
modification,  or  development  of  an  exercise  can  be  an  extremely  cumbersome 
and  time  consuming  process.  This  is  particularly  the  case  where  calculation 
of  the  developing  time  line  scenario  interaction  is  necessary  to  assure 
appropriate  scenario  events  in  support  of  the  exercise  objectives.  The 
fast-time  simulation  should  provide  capabilities  for  the  instructor  to: 

1.  call  up  an  existing  exercise  and  run  it  in  fast-time  to  any 
desired  point,  retrace  steps,  and  so  on; 

2.  investigate  alternative  actions  from  any  desired  time  to  any 
other  desired  time; 

3.  generate  and  display  relevant  performance  indicators  and 
situation  parameters  (e.g.,  target  range)  at  the  various  times  of  the  actual 
scenario  and  the  alternatives  under  investigation; 

4.  modify  any  of  the  scenario  parameters,  enabling  investigation  of 
the  tactical  parameters  and  performance  indicators  at  any  subsequent  point 
in  time; 

5.  enter  new  performance  indicator  algorithms  to  be  recorded  and 
displayed  during  or  subsequent  to  the  exercise; 

6.  insert  instructor  cues  to  be  automatically  keyed  at  times  during 
the  exercise  based  on  various  aspects  of  the  scenario  parameters  and/or  time; 

7.  configure  new  feedback  display  formats  for  trainee  briefings 
based  on  the  training  objectives,  issues  to  be  focused  on,  or  parameters 
generated  during  this  scenario,  and  so  on; 

8.  develop  new  subjective  performance  monitoring  input  categories 
for  observation  entry  by  the  instructor  during  the  actual  exercise; 

9.  setup  all  necessary  scenario  parameters  for  running  tne  scenario 
on  the  training  device; 


10.  permanently  record  the  exercise  in  the  exercise  library  for 
later  review  via  this  fast-time  simulation  capability  and  for  actual  running 
during  a  training  exercise  on  the  training  device. 

These  capabilities  are  representati ve  of  those  that  should  be 
included  in  an  exercise  development  simulation  capability.  An  investigation 
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should  be  made  to  determine  which  capabilities  would  be  most  helpful  to  the 
instructor,  and  the  specific  characteristics  of  their  interface.  The 
exercise  development  simulation  should  be  a  part  of  the  instructor  interface 
language,  discussed  above. 


d.  (84)  Training  devices  have  typically  been  poorly  human  factored  from 
the  instructor's  standpoint.  Many  of  the  relevant  human  factors-related 
issues  have  been  addressed  elsewhere  in  this  research  and  development  plan. 
However,  it  is  worthwhile  to  note  that  a  distinct  human  factors  research  ana 
development  effort  should  be  initiated,  either  under  that  heading  or 
comprised  of  various  individual  parts  such  as  broken  out  in  this  plan.  Many 
of  the  SMARTTS  capabilities  would  have  been  incorporated  into  earlier 
training  devices  if  the  instructor  and  trainee  interfaces  had  been 
adequately  looked  at  from  a  human  factors  standpoint. 

e.  (85)  Instructional  support  material  should  be  developed  to  assist 
the  instructor  in  understanding  and  conducting  an  effective  training 
process.  This  is  an  essential  part  of  every  training  system,  and  often 
overlooked  during  the  development  and  implementation  of  the  expensive 
simulator-based  hardware.  An  investigative  effort  should  be  undertaken  to 
determine  the  characteristics  of  the  essential  instructional  support 
material,  with  guidelines  developed  for  its  inclusion  in  the  procurement  of 
every  training  device/system.  A  variety  of  material  should  be  available  to 
the  instructor  as  his  primary  resource  for  developing  the  training  device 
exercise  scenario.  Most  of  this  material  would  likely  be  in  the  form  of 
handbooks,  although  some  could  be  available  in  a  computer  data  base. 
Ideally,  a  increasingly  greater  proportion  of  this  necessary  information 
would  be  incorporated  into  computer-based  processing  capabilities  over  time, 
to  more  directly  assist  the  instructor.  Information  to  be  considered  as  a 
part  of  required  instructional  support  material  would  include:  (a)  a 
complete  set  of  tactical  and  behavioral  training  objectives, 
cross-referenced  to  exercises  and  operational  situations  (this  may  include 
cross-referencing  to  the  standardized  scenarios  or  scenario  parts  as 
suggested  above);  (b)  a  complete  set  of  tactical  reference  information,  such 
as  naval  weapons  publication  (NWP)  series  documents;  (c)  a  set  of  guidelines 
regarding  training  methodologies,  training  device  operations,  and  training 
exercise  development;  and  (d)  trainee  input  characteristics  information, 
including  previous  trainee/team  performance  information,  and 
training/tactical  needs.  The  information  related  to  trainee  input 
characteristics'  could  be  available  on  a  large  information  base  assembled 
from  previous  training  exercises  of  that  particular  individual  or  team, 
together  with  comparative  data  on  the  population  of  similar  trainees,  and  so 
on.  Guidelines  should  be  developed  for  specifying  the  necessary  material  as 
a  part  of  each  training  device/system. 

f.  (86)  Artificial  intelligence  should  be  investigated  as  a  capability 
for  providing  assistance  to  the  instructor  in  conducting  his  many 
functions.  It  is  expected  that  the  artificial  intelligence  capabilities 
would  be  most  applicable  to  the  monitoring  and  control  of  the  training 
exercise,  as  opposed  to  some  of  the  other  instructor  functions.  An 
investigative  program  should  be  initiated  to  develop  and  evaluate  artificial 
intelligence  characteristics  for  instructor  support  during  the  training 
exercise. 
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g.  (87)  Instructor  console  design  features  should  be  investigated.  In 
addition  to  the  investigation  of  a  general  purpose  instructor's  console, 
research  should  be  conducted  to  evaluate  and  develop  effective  instructor 
console  design  features.  These  features  may  be  appropriate  to  a  particular 
training  situation  (e.g.,  submarine  tactics)  or  generally  applicable  across 
a  variety  of  situations.  An  example  might  be  automatic  selection  of 
feedback  displays  keyed  to  each  scenario  exercise.  Rather  than  the 
instructor  having  to  select  each  display,  perhaps  the  most  applicable 
displays  would  be  automatically  selected  by  the  computer,  or  have  been 
previously  designated  to  be  brought  up  with  each  particular  exercise. 

h.  (88)  The  effectiveness  of  alternative  training  methods  should  be 
investigated  with  regard  to  various  trainee  input  characteristics, 
particular  training  problem  areas  (i.e.,  scenario  types),  and  other  aspects 
of  the  various  training  situations.  This  type  of  research  should  be 
continuous,  since  new  training  methods  and  aids  are  continuously  being 
developed.  The  product  of  this  research  should  be  guidelines,  or  otherwise 
specific  guidance  information,  addressing  the  applicability  of  alternative 
training  methods  in  various  training  situations.  Ideally,  the  guideline 
information  would  be  specific  to  applied  areas  of  training. 

i .  (89)  The  instructor  has  been  shown  to  be  one  of  the  most  important 
components  of  the  training  system  (e.g.,  Hammell,  Gynther,  Grasso  and 
Gaffney,  1981).  Relatively  little  objective  data  is  available  on  instructor 
characteristics  and  functioning.  It  is  recommended,  therefore,  that  an 
investigative  program  be  initiated  to  determine  relevant  instructor 
characteristics,  resulting  in  guidelines  for  instructor  selection, 
preparation,  and  evaluation.  The  instructor  should  be  investigated  similar 
to  any  other  element  of  the  training  system,  with  necessary  information 
developed  to  guide  in  instructor  selection  and  training  to  assure  that  he  is 
an  effective  component  of  the  training  system.  The  instructor  is  probably 
the  most  overlooked  element  of  the  training  system,  the  one  that  has 
probably  the  greatest  impact  on  the  effectiveness  of  the  training  system, 
and  ultimately  a  high  cost  component  in  terms  of  availability  and  life-cycle 
cost. 


5. 2. 2. 7  Trainee  Specific  Characteristics.  (90)  Similar  to  that  noted 
above  for  instructor-specific  characteristics,  many  of  the  characteristics 
identified  in  other  parts  of  this  research  and  development  plan  pertain  to 
the  trainee  interface.  Several  additional  characteristics  should  be 
considered  for  investigation: 

a.  (91)  A  high  level  of  trainee  motivation  is  essential  to  an  effective 
training  process.  Various  ways  of  increasing  trainee  motivation  (e.g., 
interest)  should  be  investigated.  These  include  the  use  of  gaming 
techniques,  interesting  and  informative  information  displays,  innovative 
aids,  alternative  training  methodologies,  and  so  on.  This  investigative 
program  should  focus  on  ways  of  increasing  trainee  motivation,  and  not  ways 
of  developing  more  effective  information  presentation,  etc. 


b.  (92)  Student  participation  in  the  training  process  is  considered 
essential.  The  SMARTTS  preprototype  provides  an  entry  key  for  the  student 
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to  identify  those  times  during  the  simulator  exercise  that  he  wishes  to  key 
on  during  the  postproblem  briefing.  The  effectiveness  of  this  or  other 
types  of  trainee  entry  devices/techniques  should  be  investigated  with  regard 
to  their  impact  on  the  effectiveness  of  training. 

c.  (93)  Research  should  be  conducted  on  the  best  ways  to  tailor 
training  to  individual  students,  subteams,  or  the  complete  team.  There  has 
been  a  variety  of  research  conducted  over  the  years  on  tailoring  training  to 
individuals  (e.g.,  individual  learning  styles).  Little  of  this  research  has 
addressed  tailoring  training  to  subteams  or  teams.  Each  of  these  should  be 
looked  at.  In  particular,  methods  of  tailoring  training  on  a 
cost/effectiveness  basis  within  the  available  resources  today  should  be 
investigated.  For  example,  in  a  complex  training  situation  involving  a 
large  team  of  individuals  (e.g.,  the  submarine  combat  team)  it  may  be 
possible  to  configure  scenarios  that  provide  tailored  training  with  regard 
to  difficulty  and  presentation  style  simultaneously  to  meet  the  needs  of 
each  individual  trainee,  and  in  the  team  context.  Automated  assistance 
could  be  provided  to  the  instructor  to  help  in  configuring  a  scenario  to 
simultaneously  meet  the  multiple  needs  of  various  trainees. 

d.  (94)  The  retention  of  training  is  an  important  issue  in  most 
training  contexts,  and  one  that  has  received  relatively  little  attention. 
Considerable  effort  is  spent  in  the  retraining  of  individuals  and  teams  in 
most  aspects  of  military  operations.  The  retraining  intervals  are  usually 
chosen  on  the  basis  of  many  factors,  with  little  objective  data  available 
related  to  the  actual  need  for  retraining.  The  retention  issue  has  come  up 
in  many  contexts,  spanning  from  nuclear  power  plant  operator  to  merchant 
marine  deck  officer  to  naval  submarine  commanding  officer.  It  is 
recommended  that  an  in-depth  investigative  program  be  initiated  to 
thoroughly  investigate  the  retention  '  issue.  This  should  be  looked  at  with 
regard  to  the  specific  aspects  of  retention  relevant  to  the  requisite  s  '11s 
and  knowledge,  and  the  varying  necessary  retraining  intervals  for  eav,h. 
This  data  base  could  be  extremely  useful  in  determining  the  period  of  time 
between  retraining  sessions,  as  well  as  assist  in  appropriately  focusing 
training  during  each  retraining  session.  For  example,  it  may  be  found  that 
the  higher  level  decisionmaking  skills  are  retained  well  over  time,  while 
the  more  procedural  equipment  operating  skills  are  retained  poorly.  The 
experience  that  operators  have  in  particular  equipment  operating  skills 
during  the  interval  between  the  retraining  sessions,  however,  may  also 
differentially  impact  the  level  of  skill  retained.  Hence,  based  on  these 
factors,  for  example,  the  hypothetical  retraining  periods  would  primarily 
focus  on  those  equipment  operating  skills  that  have  received  little 
experience  during  the  intervening  operational  period,  and  in  a  much  less 
frequent  interval  address  the  high  level  decisionmaking  skills  and  those 
equipment  operating  skills  that  receive  considerable  use.  The  thrust  of 
this  research  program  should  be  to  develop  guidelines  from  which  the 
retraining  policies  could  be  developed  witn  regard  to  each  specific 
application. 

e.  (95)  The  effectiveness  of  alternative  display  characteristics  for 
effectively  conveying  information  to  trainees  during  various  phases  of  the 
training  process  should  be  investigated.  Tnis  issue  would  likely  be 
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investigated  with  regard  to  the  information  presentation  area  noted  above 
(Section  5. 2. 2. 4). 


5. 2. 2. 8  Training  System  Management.  (96)  Training  system  management  is 
one  of  the  three  major  areas  identified  as  requiring  SMARTTS-type  TAT 
characteristics;  the  other  two  are  the  instructor  interface  and  the  trainee 
interface.  Training  system  management  includes  characteristics  to  support 
the  development  and  maintenance  of  an  appropriate  training  structure  (e.g., 
setup  and  conduct  of  multiple  training  courses),  intersite  coordination, 
(i.e.,  including  exchange  of  training  performance  data,  training  exercise 
designs,  training  material,  and  so  on),  and  intrasite  coordination.  In 
addition,  training  system  management  includes  the  responsiveness  of  the 
training  facility  to  the  operational  needs  (e.g.,  interfacing  with  type 
commanders,  etc.).  The  majority  of  capabilities  necessary  to  support 
training  system  management  are  related  to  processing  support/longitudinal 
analysis  capabilities.  The  research  issue  to  be  addressed  under  training 
system  management  is  the  identification  of  characteristics  necessary  to 
support  this  major  area  of  training  system  functioning.  This  should  be  an 
investigation  to  determine  the  proper  management  role  to  be  conducted  within 
the  training  system,  including  identification  of  the  management  functions, 
interfaces,  and  so  on.  The  system  characteristics  necessary  to  support 
management  would  likely  overlap  considerably  with  those  identified  for 
processing  support  (Section  5. 2. 2. 5).  Hence,  this  research  area  and  the 
processing  support  research  area  would  likely  coincide  after  the 
requirements  are  identified. 


5. 2. 2. 9  Training  Assistance  Technology  Guidelines.  (97)  The  preprototype 
SMARTTS  fs  the  initial  implementation  of  training  assistance  technology. 
Its  evaluation  will  provide  information  relevant  to  the  evaluation  of 
training  assistance  technology  as  a  major  element  of  the  training  system, 
along  with  evaluation  of  the  specific  aspects  of  TAT  incorporated  into 
SMARTTS.  TAT  and  SMARTTS  have  evolved  from  and  extensive  analysis  the 
training  system  and  the  training  device,  resulting  in  a  series  of 
recommendations  pertaining  to  characteristics  that  should  be  added  to  many 
training  systems.  A  major  reason  why  TAT  capabilities  have  not  been 
traditionally  included  in  the  training  system  is  the  lack  of  guidance 
information  suggesting  their  need  and  effectiveness  in  the  training  process, 
and  identifying  characteristics  appropriate  to  various  aspects  of  the 
training  situation.  Guidelines  should  be  developed  for  the  inclusion  of  TAT 
characteristics  in  virtually  all  training  devices,  with  the  type  and  extent 
of  characteristics  tailored  to  the  various  factors  pertinent  to  each 
training  device-system.  The  guidelines  should  address  the  following: 

a.  A  comprehensive  list  of  TAT  characteristics  should  be  developed. 
This  list  should  identify  each  characteristic  (e.g.,  objective  Pi's; 
subjective  Pi's;  information  displays),  identify  alternative  types  and/or 
levels  of  each  characteristic,  and  provide  examples  of  appropriate  and 
successful  application  of  each  characteristic.  Information  should  be 
provided  in  the  guidelines  explaining  the  characteristic,  its  purpose  in 
training,  and  its  typical  use.  This  list  would  be  similar  to  a  shopping 
list  that  could  be  used  in  a  check-off  fashion  regarding  any  particular 
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training  device/system  and  training  application  to  achieve  a  quick 
identification  of  the  potentially  useful  TAT  characteristics. 

b.  In-depth  explanatory  information  should  be  provided  regarding  each 
TAT  characteristic.  This  information  should  address  the  range  of  design  for 
each  characteristic,  the  range  of  cost  for  development,  the  capabilities  and 
limitations  of  each  regarding  the  training  process,  and  those  factors 
impacting  the  need  for  each  TAT  characteristic.  This  type  of  information 
would  provide  general  guidance  from  which  the  appropriateness  of  a 
particular  TAT  characteristic  for  any  specific  training  application  could  be 
determined. 

c.  Suggested  design  methodologies  and  their  inherent  steps  should  be 
recommended  for  determination  of  the  appropriate  TAT  characteristics.  The 
instructional  systems  development  (ISD)  process  is  an  example  of  this  type 
of  methodology  to  be  incorporated  into  the  guidelines. 

d.  The  guidelines  should  address  the  government's  preparation  of 
requests  for  proposals,  military  characteristics,  etc.  That  is,  they  should 
provide  guidance  information  relevant  to  the  specification  of  necessary  TAT 
characteristics.  Furthermore,  the  guidelines  should  also  address  the  actual 
design  of  the  training  device/system  from  the  contractors  standpoint.  That 
is,  they  should  provide  guidance  information  relevant  to  the  development  of 
specific  design  characteristics  regarding  TAT  in  response  to  the 
requirements. 

e.  The  process  of  incorporating  TAT  characteristics  involves 
identifying  the  need  for  specific  generic  characteristics  (e.g.,  performance 
indicators,  feedback  displays),  and  tailoring  the  specific  interfaces  to  the 
particular  needs  surrounding  each  training  situation. 

f.  Guideline  information  could  be  developed  in  the  form  of  data  item 
description  documents. 

The  TAT  guidelines  should  essentially  be  a  handbook  to  be  used  by 
individuals  to  specify  characteristics  of  the  training  device/system,  and  to 
assist  in  the  design  of  the  training  device/system.  They  should  be  written 
generically,  with  specific  applied  examples.  They  should  provide 
information  which  would  assist  the  designer  in  identifying  the  specific  TAT 
characteristics  and  determining  their  particular  design. 


5.2.3  OTHER  AREAS.  The  above  two  major  areas  recommended  in  this  researcn 
and  development  plan,  21A41A  site  work  and  TAT  research  and  development, 
address  the  predominent  issues  directly  stemming  from  SMARTTS  and  relevant 
to  further  investigation.  Several  additional  research  and  development 
issues  may  be  included  under  the  rubric  of  training  assistance  technology, 
although  not  directly  impacted  by  SMARTTS.  These  are  very  briefly 
identified  in  this  section,  since  they  should  definitely  be  included  in  a 
research  and  development  program  investigating  training  assistance 
technology. 
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5.2.3. 1  Gaining.  (98)  Single  and  multiple  person  games  should  be 
investigated  and  incorporated  into  the  training  subsystem  of  training 
devices.  The  AIT  in  SMARTTS  enables  it  to  be  used  in  a  gaming  context,  if 
so  desired.  Considerable  research  into  gaming  does  exist  which  goes  well 
beyond  the  AIT  free-play  context.  The  appropriate  gaming  techniques  should 
be  brought  into  the  training  situation,  particularly  at  the  level  of 
individual  training.  The  level  of  gaming  can  encompass  a  broad  range,  from 
very  simplistic  rapid  games  to  the  more  complex  tactical  engagement  games. 
A  gaming  capability  could  be  one  additional  element  added  to  SMARTTS  that 
would  further  enhance  its  value  as  an  individual  training  device.  The 
investigative  effort  into  gaming  should  seek  to  evaluate  its  effectiveness 
at  various  levels  of  training,  determine  the  more  effective  dimensions  of 
games  that  should  be  incorporated  into  training  situations,  and  identify 
those  aspects  of  the  training  situation  amenable  to  the  use  of  gaming.  It 
is  expected  that  gaming-based  trainers  will  form  a  substantial  segment  of 
the  overall  training  system  in  the  near  future.  Unfortunately,  although 
considerable  research  has  been  accomplished  concerning  gaming,  relatively 
little  of  it  has  addressed  the  issues  regarding  training  effectiveness  and 
gaming. 


5. 2. 3. 2  Training  Methodologies.  (99)  A  wide  variety  of  training 
methodologies  currently  ex i s  t  and  are  in  practice.  These  stem  from 
classical  methods,  such  as  the  use  of  immediate  feedback,  to  contemporary 
methods  that  depend  heavily  on  automated  training  aids  (e.g.,  computer- 
assisted  instruction).  Most  training  methods  in  applied  practice  in  the 
military  environment  seem  to  challenge  the  student,  and  place  him  in  a  trial 
and  error  learning  situation;  feedback  is  typically  provided  concentrating 
on  the  student's  errors.  Although  this  general  set  of  methods  is  effective 
for  a  considerable  portion  of  training,  other  methods  are  available  which 
are  likely  to  be  substantially  more  cost-effective  for  many  areas  of 
training.  Although  considerable  research  does  exist  regarding  a  variety  of 
training  methods,  the  findings  have  not  been  adequately  transfused  into  the 
applied  training  environment.  Research  is  necessary  to  investigate 
alternative  training  methods  in  applied  training  settings.  For  example, 
certain  methods  may  be  most  appropriate  to  high  level  officer  tactics 
training  (e.g.,  delayed  feedback  positive  guidance).  Very  different 
training  methods  may  be  appropriate  to  very  basic  level  operator  training 
(e.g.,  immediate  feedback).  Ideally,  guidelines  would  be  developed  as  a 
result  of  this  investigative  program  to  assist  in  the  design  of  the  complete 
training  system,  from  the  standpoint  of  determining  the  appropriate  training 
metnods  to  be  used  during  each  stage  of  the  training  process,  as  a  part  of 
the  training  system. 


5. 2. 3. 3  Training  Effectiveness  Measurement  and  Evaluation.  (100)  The 
measurement  and  evaluation  of  training  effectiveness  snould  be  an  ongoing 

practice  for  every  training  system.  The  evaluation  should  uegin  at  trie 

design  stage  of  the  particular  training  system,  follow  tnrougn  tne 
development  and  implementation  stage,  and  continue  for  as  long  as  tne 

training  system  is  in  operation  and  can  be  cost/effect i vely  modified.  Tne 
intent  of  this  evaluation  should  be  to  insure  (a)  that  appropriate  TAT  and 

other  necessary  character istics  are  induced  in  tne  design  and  adequately 
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developed;  (b)  to  empirically  verify  the  effectiveness  of  the  various  design 
characteristics  with  regard  to  modification  of  that  particular  training 
system  and  future  training  system  designs;  and  (c)  to  continually  upgrade 
the  effectiveness  of  the  particular  training  system  while  in  use.  Training 
effectiveness  measurement  and  evaluation  is  occasionally  conducted,  usually 
from  a  purely  empirical  research  standpoint.  Seldom,  if  ever,  is  the 
training  system  design  evaluated  from  the  standpoint  of  its  potential 
training  effectiveness.  As  the  hardware  and  software  cnaracteri sties  are 
evaluated  at  various  stages  during  the  design  and  development  process  with 
regard  to  their  meeting  design  goals,  so  should  the  training-related 
characteristics  of  the  training  system  be  evaluated  with  regard  to  their 
meeting  training  effectiveness  goals.  SMARTTS  developed  an  initial  cut  at 
this  type  of  a  process  in  terms  of  the  training  system  evaluation  test 
(TSET)  which  was  conducted  at  the  same  time  as  the  factory  acceptance  test. 
The  TSET  was  an  initial  cut  in  that  it  took  place  long  after  the  system  was 
designed,  and  sought  to  develop  an  approach  to  the  evaluation  of  potential 
training  effectiveness.  Its  primary  purpose  was  to  identify  the  myriad  of 
issues  surrounding  the  evaluation  of  training  effectiveness  during  the 
design  process,  and  suggest  alternative  means  for  that  type  of  evaluation. 
An  effort  should  be  initiated  to  develop  requirements  for  the  training 
effectiveness  evaluation  at  various  stages  during  the  design  and  development 
process  of  the  training  system.  The  research  area  briefly  discussed  above 
concerning  TAT  guidelines  (Section  5. 2. 2. 9)  would  be  an  important  source  of 
information  for  this  evaluation  instrument. 


The  effectiveness  of  each  training  system  after  it  is  installed  should 
be  evaluated  using  objective  data  where  possiole.  The  intent  should  be  to 
verify  the  effectiveness  of  the  particular  design,  and  identify  strong  and 
weak  characteristics  for  consideration  in  future  designs.  This  evaluation 
could  be  accomplished  in  a  variety  of  ways  including  evaluation  on  the 
device  itself  and  via  transfer  of  training  investigation.  Evaluation 
procedures  and  tools  should  be  developed  to  provide  a  generic  framework  for 
facilitating  the  evaluation  of  each  training  device.  This  developmental 
program  would  have  to  devise  guidelines  for  the  specification  of  specific 
performance  measures,  data  collection  tools,  data  collection  and  analysis 
procedures,  evaluation  procedures,  and  summary  conclusion  formats  for 
feedback  to  the  general  training  system  design  community.  The  intent  shoulo 
be  to  develop  generic  tools  that  can  be  adapted  to  the  specific  needs  of 
particular  applications  (e.g.,  funding  availability,  time/senedu le 
constraints,  complexity  of  training,  etc.). 

Every  time  a  training  exercise  is  run  on  a  sopn isticated  simulator-based 
training  device  considerable  performance  data  is  generated,  and  usually 
lost.  These  data,  particularly  when  supported  by  SMARTTS-type  TAT 
characteristics,  can  be  retrieved  and  used  as  a  major  source  of  information 
to  continually  evaluate  the  effectiveness  of  the  training  system.  This 
evaluation  should  focus  on  the  variety  of  aspects  of  the  training  system, 
including  the  training  methods,  simulation  fidelity,  the  TAT  capabilities, 
the  instructor,  and  so  on.  The  evaluation  should  be  aimed  at  identifying 
those  aspects  of  the  training  system  that  could  be  improved,  and  recommend 
improvements.  If  the  tools  are  adequately  developed  for  the  initial 
ef feet i veness  evaluation  of  the  training  system  they  could  be  also  used  for 
the  continual  evaluation. 


AD-A179  679  SUBMARINE  ADVANCED  REACTIVE  TACTICAL  TRAINING  SVSTEN 

(SNARTTSXU)  ECLECTECH  ASSOCIATES  INC  NORTH  STONINGTON 
CT  T  J  HAHHELL  SEP  82  EA-82-U-076 
UNCLASSIFIED  NAVTRAEQUIPC-80-C-0079  N61229-80-C-0079  F7G  579 


NAVTRAEQUIPCEN  80-C-0079 


The  development  of  evaluation  instruments  and  procedures  is  a  necessary 
requirement  to  assure  the  effective  design  and  evolution  of  training 
systems.  This  has  been  generally  overlooked  from  the  standpoint  of 
day-to-day  applied  training  operations,  and  usually  reserved  for  occasional 
laboratory  investigation.  As  with  any  complex  system,  it  is  necessary  to 
have  a  constant  source  of  information  feedback  regarding  performance  to 
assure  that  the  system  functions  properly,  and  to  evolve  its  capabilities  as 
the  training  needs  change  and  as  new  methods  become  available. 


5. 2. 3. 4  Embedded/Organic  Training.  (101)  Embedded/organic  training  is 
likely  to  become  the  predominant  context  in  which  training  is  provided  to 
individuals  and  teams  in  complex  systems.  This  may  hold  true  for  both 
at-sea  training  as  well  as  shore-based  training.  Modern  computer  technology 
allows  the  integration  of  training  capabilities  within  operational  systems 
to  take  advantage  of  available  over  time  at  sea  for  training.  Furthermore, 
the  future  weapon  systems  are  becoming  increasingly  complex  and  expensive, 
requiring  a  large  number  of  operators.  The  expense  of  placing  operational 
system  simulators  in  shore-based  facilities  will  decrease  the  number  of  such 
facilities.  Hence,  a  greater  portion  of  training  is  likely  to  shift  to  the 
at-sea  environment.  Furthermore,  shore-based  training  facilities  are  likely 
to  rely  heavily  upon  the  operational  system  itself;  when  an  embedded 
training  capability  exists,  it  is  likely  that  the  shore-based  facility  will 
simply  use  that  capability  in  place  of  other  unique  training  capabilities. 
Hence,  the  design  of  the  embedded/organic  training  system  that  will  be 
integrated  into  future  weapon  systems  is  likely  to  become  the  predominent 
future  training  systems. 

Although  TAT  capabilities  are  extremely  important  for  the  shore-based 
training  systems,  as  has  been  demonstrated  in  several  experiments,  they  are 
even  more  important  for  embedded/organic  training  systems.  The  snore-based 
facilities  are  devoted  solely  to  training,  with  a  full-time  instructional 
staff,  and  other  support  capabilities.  Conversely,  the  embedded/organic 
training  system  is  merely  a  minor  part  of  the  complex  weapon  system. 
Furthermore,  full-time  instructional  personnel  do  not  normally  exist  in  the 
at-sea  training  situation.  Nor  is  the  focus  of  that  operational  system 
primarily  on  training.  Hence,  the  general  at-sea  environment  for 
embedded/organic  training  will  be  much  less  conducive  to  the  conduct  of 
effective  training  than  are  the  existing  shore- based  facilities.  The  need 
for  TAT  characteristics,  therefore,  is  extremely  important. 

The  embedded/organic  training  environment  requires  TAT  characteristics 
to  be  directed  at  the  individual  trainee  so  as  to  assist  nim  in  directing 
his  own  training,  generating  his  own  feedback  information  from  the  system, 
evaluating  nis  own  progress,  and  so  on.  This  would  also  be  true  of  subteains 
and  teams.  The  need  for  automated  performance  indicators  and  capable 
feedback  displays  becomes  extremely  important,  since  these  will  become  the 
primary  means  by  which  the  essential  element  of  feedbacK  is  provided  to  the 
trainees.  It  is  expected  that  the  TAT  capabilities  required  in 
emoedded/organic  training  are  much  more  critical  then  those  required  in 
shore-based  training. 
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An  investigative  effort  should  be  initiated  to  determine  the  TAT 
capabilities  necessary  to  support  embedded/organic  training,  across  the 
various  platforms  in  which  such  training  will  be  implemented.  Empirical 
investigations  should  follow  on  prototype  units  to  investigate  cornerstone 
capabilities  of  TAT  in  this  environment.  Guidelines  should  then  be 
developed  to  provide  necessary  information  to  assist  the 
designers/developers  of  embedded/organic  training,  and  eventually 
appropriate  guidelines  and  other  training  materials  should  be  developed  to 
support  the  users  in  conducting  an  effective  training  process.  The  general 
approacn  should  be  to  automate  as  many  of  the  TAT  capabi 1 ities  as  possible 
(e.g.,  trainee  guidance  information) .  Several  of  the  characteristics  noted 
in  this  research  and  development  plan  would  likely  have  a  greater  need  for 
incorporation  into  embedded/organic  training  than  into  shore-based  training 
(e.g.,  gaming  models,  such  as  those  based  on  the  AIT).  The  concepts  of  TAT 
are  similar  for  embedded/organic  training  as  they  are  for  shore-based 
training  such  as  SMARTTS.  The  particular  TAT  capabilities  required  most, 
and  their  specific  characteristics,  are  however  expected  to  differ  in  the 
embedded/organic  situation,  embedded/organic  training  is  rapidly  becoming  a 
reality,  and  should  pose  a  major  training  problem  for  the  fleet.  TAT 
capabilities  are  essential  to  insure  effective  training.  Unfortunately,  the 
research  data  is  minimal  in  this  area. 


5.3  RESEARCH  AND  DEVELOPMENT  PLAN  SUMMARY 


Tnis  research  and  development  plan  addresses  the  three  major  work  areas 
of  21A41A  site  work,  TAT  research  and  development,  and  other  areas  of 
research  and  development.  The  purpose  of  the  plan  is  to  identify  the  major 
research  and  development  issues  relevant  to  training  assistance  technology 
(TAT),  and  the  SMARTTS  implementation  of  TAT.  The  plan  seeks  to  address  the 
complete  range  of  relevant  research  and  development  issues,  summarizing  the 
status  of  each.  Those  issues  pertaining  to  the  2 1 A4 1 A  site  work  are 
addressed  at  the  very  detailed  level,  since  recommendations  can  be  made 
regarding  specific  aspects  of  the  SMARTTS  preprototype.  Other  more  general 
issues  are  also  addressed  regarding  the  site  work.  The  issues  addressed 
under  TAT  research  and  development  are  typically  done  so  at  a  broader  level, 
since  they  are  not  necessarily  in  reference  to  a  particular  installation  or 
device  but  rather  with  regard  to  generic  TAT.  Nevertheless,  specific 
aspects  are  addressed  regarding  several  of  the  issues  as  appropriate.  The 
researcn  issues  addressed  under  the  "other  areas"  category  are  done  so 
briefly,  since  these  are  broad  issues  relevant  to  TAT,  but  not  necessarily  a 
direct  part  of  TAT.  Tne  intention  with  regard  to  "other  areas"  has  been  to 
identify  the  major  issues,  and  to  provide  some  insight  regarding  each. 
Obviously,  a  myriad  of  specific  issues  pertain  to  each  of  these. 

A  broad  range  of  research  and  development  issues  have  been  identified, 
clearly  substantially  more  than  can  be  accomplished  over  a  reasonable  period 
of  time.  Several  of  these  issues  pertain  to  tne  estao I ishment  of  long-term 
programs,  some  of  which  will  provide  immediate  and  ongoing  benefits  wnile 
others  are  likely  to  provide  only  long-term  benefits.  Other  issues  are 
hignly  specific  and  are  expected  to  provide  immediate  results.  This  plan 
addresses  the  broad  range  of  issues,  from  wnich  a  next-level  more  detailed 
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research  and  development  plan  should  be  constructed.  The  next-level  of  plan 
should  select  those  issues  that  appear  to  have  good  potential  for 
investigation  and/or  accomplishment.  To  assist  in  the  development  of  the 
next-level  plan  several  of  the  major  considerations  likely  to  impact  the 
development  of  this  next  iteration  are  addressed  in  Table  1.  This  table 
provides  information  estimates  concerning  each  research  issue,  pertaining  to 
the  relevant  dimensions  identified  earlier.  An  explanation  of  the 
categories  and  rationale  pertaining  to  each  subcategory  in  Table  1  is 
presented  below. 

a.  Location  —  Operational  training  device  site  and/or  laboratory.  The 
laboratory  should  be  viewed  as  the  location  to  perform  most  initial 
research,  identifying  those  candidate  issues/characteristics  for  which  good 
evidence  suggests  will  substantially  impact  training  cost/effectiveness. 
The  laboratory  should  generate  the  concepts,  perform  initial  evaluation  of 
them,  and  act  as  a  screening  process  to  identify  those  that  have  the 
greatest  potential  to  improve  the  training  process.  Generally,  this  would 
include  all  basic  research  and  much  applied  research.  The  site  should  be 
used  to  evaluate  those  issues/characteristics  that  have  a  high  immediate 
potential  for  improving  the  cost/effectiveness  of  the  training  process, 
based  on  laboratory  data  and/or  training  analyst  opinion.  The  site  should 
also  be  used  for  providing  modifications  and/or  upgrading  of  current  TAT 
capaoilities  already  implemented  via  SMARTTS.  When  the  need  for  test  and 
evaluation  in  the  operational  training  system  exists,  particularly  with 
regard  to  newly  implemented  characteristics,  it  should  be  conducted  at  a 
site. 


The  first  two  columns  of  Table  1  address  the  location,  identifying 
that  the  particular  issue  should  be  addressed  at  the  site  or  in  the 
laboratory.  Several  issues  are  indicated  for  addressing  at  both  the  site 
and  in  the  laboratory  setting.  Work  regarding  these  issues  may  be 
appropriate  in  both  work  areas,  depending  upon  the  particular  aspects  being 
addressed  and  the  phase  of  work. 

b.  Work  Type  --  Basic  research,  applied  research,  system  modification, 
or  system  evolution. 

1.  Basic  Research  --  Research  dealing  with  fundamental,  generic 
training  issues,  not  tied  to  a  specific  context. 

2.  Applied  Research  —  Research  that  is  generally  context-specific 
to  a  particular  applied  training  environment. 

3.  System  Modification  --  Modification  or  refinement  of  existing 
TAT/SMARTTS  capabilities  on  the  2 1 A4 1 A  to  improve  their  functioning. 

4.  System  Evolution  --  The  development  of  new  TAT  capabilities  with 
regard  to  the  2 1 A4 1 . 

The  basic  and  applied  research  types  of  worx  are  eitner  general,  or 
may  pertain  to  one  or  more  different  training  devices,  including  the  2 1 A4 1 A 
SCSI.  Tne  system  modification  and  system  evolution  types  of  work  both 
pertain  to  the  2 1 A4 1 A  SCST.  Hence,  most  of  the  "21A41A  site  worx"  coincides 
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with  the  last  two  respective  Work  Types,  while  much  of  the  "TAT  research  and 
development"  and  "other  areas"  coincide  with  the  first  two  types  of  work. 
Usually  only  one  category  of  Work  Type  is  indicated  for  most  of  the  research 
and  development  issues,  although  several  Work  Types  are  indicated  as 
appropriate  on  some  of  the  issues. 

c.  Application  --  Training  process,  or  support.  Those  research  and 
development  issues  that  would  directly  affect  conduct  of  the  training 
process  fall  under  the  training  process  category  (e.g.,  performance 
indicators,  feedback  displays).  Those  research  issues  that  address  training 
process  development,  system  management,  etc.,  do  not  have  a  direct  impact  on 
the  conduct  of  the  training  process  itself,  and  are  indicated  as  support. 
Only  four  issues  impact  both  training  process  and  support  functions. 

The  application  category  provides  information  relevant  to  the 
evaluation  of  the  priority  to  be  given  to  each  research  issue.  However,  a 
direct  relationship  between  application  and  priority  was  not  determined. 
Certain  support  processes,  such  as  processing  capabilities  to  assist  in 
development  of  training  exercises,  may  have  a  substantial  impact  on  the 
effectiveness  of  the  training  process,  while  other  capabilities  which 
directly  supporting  the  training  process  may  have  a  minimal  impact  on  the 
training  effectiveness. 

d.  Priority  --  Three  levels  of  priority  were  allocated: 

(H)  --  High  priority 

(M)  --  Medium  priority 

(L)  --  Low  priority 

The  priority  afforded  the  various  research  issues  is  extremely 
important  since  it  determines  those  issues  that  will  receive  emphasis  and 
funding,  and  hence  those  that  will  be  addressed.  Many  factors  must  be 
considered  in  determining  the  priority  of  research  and  development  issues, 
including  the  potential  impact  of  results  on  improving  the  cost  and/or 
effectiveness  of  training;  the  motivational  value  to  either  the  instructor 
or  the  students;  the  cost  of  conducting  the  necessary  research  and 
development  efforts,  and  their  resulting  implementation  cost;  and  the 
climate  ( i .  e . ,  general  acceptability  within  the  Navy)  for  conducting 
particular  research  and  development  efforts.  The  priority  assigned  in  Table 
1  is  an  indication  of  the  author's  estimated  priorities,  based  on  these  and 
other  factors. 

e.  Cost  --  Several  cost  categories  are  provided  as  follows: 

( I )  --  Less  tnan  $25K 

(2)  —  $25K  -  $  1 00K 

(3)  —  $100K  -  $400K 

(4)  —  $400K-  $800K 

(5)  --  Greater  than  $800K 

These  five  cost  categories  were  set  up  to  provide  a  general 
indication  of  the  cost  nagnicii;  fir  each  of  the  research  and  development 
issues.  Obviously,  many  factors  will  affect  the  actual  cost  of  conducting 
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work  regarding  any  of  the  research  and  development  issues,  which  can 
certainly  change  the  cost  categories  as  estimated  herein.  The  estimate  of 
cost  provided  in  Table  1  was  independently  determined  for  each  research  and 
development  issue.  If  several  issues  are  combined  into  a  single  program, 
the  cost  may  change  considerably  and  be  considerably  different  than  the  sum 
of  costs,  either  up  or  down.  Furthermore,  the  scope  and  depth  of  work  to  be 
accomplished  regarding  each  research  issue  can  substantially  change  its  cost 
either  up  or  down.  Hence,  the  cost  estimates  are  provided  only  to  give  a 
ball-park  idea  of  the  relative  magnitude  of  the  various  research  and 
development  efforts. 

Usually,  a  single  cost  estimate  is  provided.  For  several  issues,  a 
range  of  costs  is  indicated  (e.g.,  1-3  indicating  the  cost  can  vary 
substantially  over  that  range  depending  upon  the  specific  factors  associated 
with  the  particular  work  effort).  The  cost  indicated  is  generally  that 
estimated  to  adequately  address  the  research  and  development  issue  within 
the  scope  intended  in  this  research  and  development  plan.  For  those  issues 
that  will  require  a  long-term  program,  usually  indicated  in  the  comments 
column,  the  costs  are  indicative  of  a  one  year  effort;  obviously,  a 
multiyear  program  will  have  costs  that  typically  vary  substantially  from 
year  to  year,  such  as  initially  low  cost  during  the  early  analysis  phase 
with  substantially  higher  cost  during  the  later  applied  developmental 
phases.  Hence,  the  cost  estimate  in  these  cases  would  represent  an  average 
level  of  effort  for  any  given  year  during  that  program. 

Several  research  and  development  issues  for  which  the  cost  was 
indicated  in  the  lowest  category  (i.e.,  1)  may  cost  substantially  less  than 
the  upper  $25!C  limit  of  that  category.  This  is  particularly  true  for 
several  of  the  issues  under  21A41A  site  work,  especially  if  several  were  to 
be  accomplished  simultaneously.  For  example,  if  five  or  six  recommended 
modifications  were  to  be  accomplished,  with  each  of  these  falling  in  the  #1 
cost  category,  the  cost  may  be  substantially  less  than  totaling  the  $25K 
value  for  each  issue.  Similarly,  cost  reductions  would  generally  occur  if 
multiple  issues  were  to  be  investigated  in  a  single  program.  This  is  due  to 
the  existence  of  many  administrative  and  logistics  cost  items  that  would  not 
change  substantially  if  multiple  efforts  were  to  be  conducted. 
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APPENDIX  A 

INSTRUCTOR  FUNCTIONS 
(An  excerpt  taken  from  Hammell,  1982) 


The  major  instructional  functions  supported  by  the  preprototype  SMARTTS 
are  as  follows: 

a.  exercise  development 

b.  montw’r  and  control  of  the  training  process 

c.  briefing  (pre,  freeze,  post) 

d.  training  system  management 

Monitor  and  control  of  the  training  process  is  the  instructor  function 
given  the  greatest  recognition  by  training  device  builders  (Figure  A-l). 
When  a  training  device  is  employed,  the  instructor  must  set  up  the  exercise, 
control  the  device  during  the  exercise  (e.g.,  maneuvering  targets),  monitor 
the  trainees'  activities,  and  provide  some  amount  of  training  during  the 
exercise  (e.g.,  guidance  and  feedback  to  the  trainees).  Capabilities,  in 
one  form  or  another,  are  provided  on  most  training  devices  to  enable  set  up 
and  control  of  the  exercise.  Some  capabilities  are  often  provided  to  permit 
monitoring  of  trainee  activities,  and  often  some  recording.  Occasionally, 
capabilities  are  available  for  providing  some  training  assistance,  such  as 
feedback.  However,  all  too  often  the  monitoring  and  control  capabilities 
installed  on  a  training  device  are  designed  from  an  engineering-control 
standpoint,  rather  than  from  an  instructional  process  control  viewpoint. 
Specific  instructor  tasks,  A-L,  are  included  in  Figure  A-2,  along  with 
generic  training  issues  and  considerations  under  each  task.  Also,  the 
multiplicity  of  branching  paths  between  tasks  is  also  indicated  to  some 
extent  by  the  A  and  B  symbols.  The  information  contained  in  Figure  A-3  is  a 
summary  of  the  type  of  analysis  information  upon  which  the  SMARTTS 
characteristics  were  based.  For  example,  the  instructor  has  a  task  to 
monitor  the  scenario  (i.e..  Task  C).  A  consideration  under  that  task, 
particularly  in  a  complex  system,  is  providing  cues  to  the  instructor 
regarding  current  and  upcoming  scenario  events.  Capabilities  were  developed 
in  SMARTTS  to  provide  cues  and  alerts  to  the  instructor  regarding  various 
tactical  actions  on  the  part  of  the  target,  as  well  as  when  various 
performance  indicators  (e.g.,  probability  of  ownship  counterdetection)  went 
beyond  preset  standards.  These  capabilities,  which  are  easily  provided  by 
the  computer-controlled  training  device,  reduce  the  instructor's  scenario 
monitoring  load,  enabling  him  to  devote  a  greater  proportion  of  time  to 
monitoring  the  trainees  (i.e..  Task  H).  Other  examples  are  given  later 
under  Training  Assistance  Technology. 

Briefing  of  the  trainees  (Figure  A-4),  in  its  various  forms,  is  probably 
the  most  important  function  performed  by  the  instructor.  In  this  function 
the  instructor  has  a  direct  interface  with  the  trainees,  and  provides  them 
with  specific  information  to  reinforce  and/or  modify  their  behavior.  In  a 
simulator-based  training  system  briefings  may  be  given  prior  to,  during  a 
pause  in,  and  following  completion  of  the  session  on  the  simulator  (i.e.. 
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pre,  freeze,  and  post  briefings,  respectively).  Although  essential  to  the 
effecti veness  of  the  training  process,  the  emphasis  placed  on  oriefings 
varies  considerably  across  training  establishments.  Furthermore,  the 
capabilities  provided  as  part  of  the  training  device/training  system  to 
assist  in  conducting  effective  briefings  are  often  quite  limited.  Examples 
of  capabilities  to  support  briefing  tasks  will  be  provided  later  below  under 
Training  Assistance  Technology. 

Exercise  development  (Figure  A-2),  which  precedes  the  conduct  of 
training  sessions,  consumes  a  substantial  portion  of  an  instructor's  time. 
This  is  a  complex  function  wherein  the  instructor  actually  develops  the 
training  program  and  its  supporting  materials.  It  is  complex  in  that  it 
often  requires  the  creative  design  of  the  course  and  its  training  exercises, 
with  the  instructor  drawing  upon  the  state-of-the-art  in  training 
methodology  to  achieve  an  effective  training  process.  Whereas  some 
capabilities,  although  usually  quite  limited,  are  often  provided  on  the 
training  device/training  system  to  support  exercise  monitoring,  control  and 
briefing,  usually  little  if  any  capabilities  are  provided  to  support 
exercise  development.  It  is  the  exercise  development  function, 
interestingly,  that  determines  how  the  training  device  will  be  employed 
within  the  training  system.  Examples  of  system  capabilities  to  support 
exercise  development  are  presented  later  under  Training  Assistance 
Technology. 

Training  system  management,  similar  to  exercise  development,  is 
typically  an  overlooked  function  of  the  instructor,  and  one  with  which  he 
spends  substantial  time.  A  wide  variety  of  tasks,  issues,  and 
considerations  may  be  associated  with  training  system  management. 
Furthermore,  each  of  these  may  depend  on  the  particular  training 
establishment  under  which  the  training  device/training  system  operates. 
Nevertheless,  several  training  system  management  tasks  are  common  across 
training  establishments,  as  indicated  in  Figure  A-5.  Perhaps  the  most 
important  common  set  of  tasks  involves  monitoring  the  effectiveness  of 
training  over  time.  For  example,  it  is  highly  desirable  to  identify  those 
training  objectives  (i.e.,  with  associated  exercises)  that  the  typical 
trainees  can  readily  perform  upon  entering  the  training  program;  and 
likewise  to  identify  those  training  objectives  (and  exercises)  that  the 
trainees  typically  have  substantial  difficulty  with,  even  near  the  end  of 
the  training  program.  Ideally,  reduced  emphasis  would  be  placed  on  the 
former  while  increased  emphasis  would  be  placed  on  the  latter  training 
objectives  in  subsequent  training  programs.  For  another  example, 
alternative  training  methods  and  training  materials  should  be  periodically 
evaluated  to  continually  upgrade  their  quality  and  the  effectiveness  of  the 
training  process.  These  are  major  management  tasks  directly  impacting  the 
cost/effectiveness  of  training,  and  subsequently  the  operational  readiness 
and  effectiveness  of  weapon  systems. 

Each  time  a  training  exercise  is  run  on  the  simulator  valuable  training 
performance  data  is  generated.  These  data  could  be  collected  over  time  and 
used  as  the  basis  for  the  above  evaluations,  and  other  training  system 
development  activities.  These  data  would  be  extremely  useful  to  most  levels 
of  management  for  evaluation  and  planning  purposes.  Relatively  little  of 
these  data  are  recorded  and  used  today  witn  regard  to  most  training 


NAVTRAEQUIPCEN  80-00079 


device/training  systems.  This  is  one  example  of  a  set  of 
device/training  system  capabilities  that  could  substantially 
training  system  management. 
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APPENDIX  B 

SMARTTS  SOFTWARE  DESCRIPTION 

The  SMARTTS  software  has  three  major  parts:  (a)  the  main  body  of 
SMARTTS  software  running  on  the  SMARTTS  host  computer  (VAX  11/780)  with  the 
VAX/VMS  operating  system;  (b)  SMARTTS  modifications  to  the  MSCG  software  to 
achieve  the  necessary  interaction  between  the  SMARTTS  host  computer  and  the 
MSCG;  and  (c)  software  modifications  to  the  data  gathering  module  of  the 
MK 117  fire  control  system  to  supply  necessary  information  to  the  SMARTTS 
host  computer.  This  appendix  summarizes  the  17  major  parts  of  SMARTTS 
software  running  on  the  SMARTTS  host  computer  comprising  part  one  and  the 
majority  of  SMARTTS  functioning. 

The  high  level  SMARTTS/VAX  System  Control  Configuration  is  illustrated 
in  Figure  Bl,  consisting  of  three  major  components.  The  SMARTTS  system 
supervisor  component  (SY)  creates,  initializes  and  terminates  subsystems; 
controls  the  system  conf iguration;  and  logs  and  distributes  alarms.  The 
system  supervisor  component  interfaces  with  a  subsystem  supervisor  component 
which  in  turn  controls  the  (a)  model  deck  subsystem,  which  is  off-line;  (b) 
classroom  subsystem  supporting  the  instructor's  console  and  displays;  and 
(c)  attack  center  subsystem  supporting  the  instructor's  console  and  two 
overhead  displays.  The  subsystem  supervisor  creates  and  terminates  all 
other  components,  implements  operating  state  changes,  and  takes  error 
recovery  action.  The  model  deck  subsystem  interacts  via  the  model  deck 
controller  component.  The  classroom  and  attack  center  subsystems  interact 
via  the  console  controller  component  (i.e.,  instructor  consoles).  An 
overview  diagram  of  these  major  components  is  presented  in  Figure  B-l. 

A  brief  description  of  each  of  the  17  major  SMARTTS/VAX  components 
follows: 

System  Supervisor  --  Creates,  initializes,  and  terminates  the  existence 
of  all  other  SMARTTS/VAX  software  components.  It  includes  monitoring  the 
availability  of  all  SMARTTS  hardware  components  and  partitioning  SMARTTS 
hardware  components  among  SMARTTS/VAX  software  subsystems.  It  is  a 
collection  of  command  files  and  programs  that  is  responsible  for  setting  up 
the  system  and  invoking  the  subsystems. 

Subsystem  Supervisor  —  A  collection  of  subroutines  that  forms  a  process 
which  is  responsible  for  controlling  a  subsystem.  It  causes  the  creation, 
ini tial ization,  and  termination  of  all  software  components  that  comprise  a 
SMARTTS/VAX  subsystem  (i.e.,  there  are  three  SMARTTS/VAX  subsystems--model 
deck,  classroom,  and  attack  center  subsystems).  Each  subsystem  is  comprised 
of  a  dedicated  set  of  SMARTTS  hardware  .  resources  together  with 
facts-resident  software  components.  Performance  of  the  subsystem  supervisor 
function  includes  monitoring  and  control  of  subsystem  operating  states. 
Each  subsystem  operating  state  determines  a  set  of  training  assistance  ana 
operational  features  that  may  be  utilized  by  an  instructor  or  system 
operator.  Performance  of  the  subsystem  supervisor  function  adjusts 
subsystem  operations  in  response  to  hardware  and  software  errors  affecting 
the  subsystems  training  assistance  and  operational  capabilities. 
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SY  —  System  Supervisor  Component 

SB  —  Subsystem  Supervisor  Component 

CC  —  Console  Controller 

SC  —  System  Controller 

MDS  —  Model  Deck  Subsystem 

CRS  --  Classroom  Subsystem 

ACS  —  Attack  Center  Subsystem 


Figure  B-l.  Major  SMARTTS/VAX  system  control  configuration 
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Console  Controller  —  It  enables  instructors  in  the  attack  center 
subsystem  and  in  the  classroom  subsystem  to  direct  respective  subsystem 
operations.  The  console  controller  communicates  with  the  instructor  by 
manipulating  the  instructors  console  plasma  entry  device.  Upon  receipt  of 
instructor  directions  the  console  controller  ascertains  the  validity  of 
instructor  directions  and  passes  such  directions  on  to  other  SMARTTS/VAX 
subsystem  software  components. 

System  Controller  —  It  allows  the  system  operator,  using  the  VAX 
VT-100,  to  direct  model  deck  subsystem  operations.  The  system  supervisor 
accepts  system  operator  directions,  checks  the  validity  of  such  directions, 
and  forwards  such  directions  to  remaining  SMARTTS/VAX  subsystems  software 
components. 

Simulation  Recorder  --  It  calculates  and  records  the  data  for  scenario 
preview  or  exercise  control  (i.e.,  ship  movements  and  performance).  The 
software  is  used  in  two  manners:  (a)  as  a  separate  process  during  scenario 
preview  to  record  SMARTTS/VAX-based  simulated  exercise  data,  and  (b)  during 
exercise  control  to  record  MSCG-based  real-time  trainer  exercise  data.  The 
exercise  data  is  recorded  to  support  the  subsequent  review  of  exercises  in 
the  attack  center  or  in  the  classroom  by  trainee  teams. 

Review  Controller  —  It  enables  the  instructor  to  retrieve  and  to  replay 
previously  recorded  exercise  data.  It  handles  the  opening  of  an  old  data 
base  for  replay  during  playback,  handles  the  movement  of  time,  and  hence  the 
moving  of  pointers  in  the  data  base. 

Display  Module  —  It  enables  the  display  of  information  on  the  various 
color  displays  (i.e.,  instructor  consoles,  overhead  CRTs  in  the  attack 
center,  and  large  screen  display  in  the  classroom).  This  component  is 
subdivided  into  two  major  parts:  (a)  SMARTTS  display  format  algorithm 
software,  and  (b)  software  that  communicates  with  the  display  processor 
device.  When  these  two  parts  are  put  together,  they  act  as  a  process  that 
controls  a  single  color  monitor. 

Automatic  Interactive  Target  (AIT)  —  The  AIT  component  is  a  collection 
of  files  that  when  linked  together  form  the  AIT  target,  providing  means  for 
automatic  control  of  an  MSCG  target.  Based  on  a  set  of  characteristic 
specification  determined  by  the  instructor,  the  AIT  function  directs  the 
actions  of  an  MSCG  target  interactively  with  the  MSCG  ownship.  The 
automatic  interaction  is  dependent  upon  ownship  actions  and  the  target 
characteristics  specification.  The  AIT  controller  is  available  during  the 
combined  SMARTTS/MSCG  operation,  as  well  as  during  standalone  SMARTTS/VAX 
exercises. 

Simulation  Controller  —  It  coordinates  the  simultaneous  executions  of 
the  SMARTTS/VAX  and  MSCG  components  during  trainer  exercises.'  It  handles 
the  control  of  the  vehicle  simulation  when  running  in  either  scenario 
preview  or  exercise  control  (i.e.,  either  joined  with  the  MSCG,  or  decoupled 
from  the  MSCG).  The  simulation  controller  receives  exercise  data  from  the 
MSCG  or  from  SMARTTS/VAX  simulation  calculation  programs  and  forwards  such 
data  to  the  simulation  recorder  for  storage  in  the  recorded  exercise  file. 
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It  reads  and  interprets  the  English-like  script,  and  directs  subsystem 
actions  using  the  scenario  profiles. 

Alternative  Tactics  Calculator  —  It  projects  the  values  of  tactical 
variables  and  performance  indicators  during  alternative  tactics  sessions. 
To  accomplish  this,  it  operates  on  data  given  to  it  by  the  console  control, 
using  the  data  calculator  and  data  manager  modules  as  necessary. 

Data  Manager  --  This  is  the  data  base  manager  for  the  SMARTTS  system. 
It  includes  an  extensive  data  definition  scheme.  It  is  used  by  other 
functions  to  store  and  subsequently  retrieve  all  relevant  SMARTTS/VAX,  MSCG, 
and  MK 117  exercise  data.  It  is  used  by  the  simulation  recorder  to  record 
trainer  exercise  data;  the  review  controller  to  retrieve  exercise  data 
during  exercise  reviews;  and  the  display  module  to  present  the  exercise  data. 

Profile  Manager  --  It  conducts  the  reading  and  rewriting  ( i . e . ,  saving 
initial  conditions)  of  the  English-like  script  that  is  used  to  control  the 
SMARTTS  system.  It  causes  scenario  data  to  be  recorded,  manipulated, 
retrieved,  and  archived  for  future  use.  It  allows  the  instructor  to 
automatically  initialize  the  SMARTTS/VAX  system  and  the  MSCG  for  a  trainer 
exercise  scenario.  It  also  allows  the  instructor  to  lay  out  an  exercise 
scenario  prior  to  its  execution,  and  to  cause  automatic  execution  of  the 
exercise  scenario.  The  simulation  controller  and  the  review  controller 
utilize  profile  manager  services  to  perform  these  functions. 

Data  Calculator  --  It  performs  calculations  to  generate  performance 
indicators  on  the  basis  of  received  data.  It  receives  exercise  data  from 
the  MSCG  and  MK117,  uses  that  data  to  calculate  the  various  parameters,  and 
passes  the  calculated  values  to  other  software  components  for  subsequent 
manipulation  and  recording.  It  also  allows  the  SMARTTS/VAX  system  to 
execute  simulated  exercises  (e.g.,  during  exercise  preview  in  the  classroom) 
via  simplified  models  in  place  of  the  MSCG  and  MK117  data  (e.g.,  own  ship 
maneuvers,  sonar  parameters,  target  motion  analysis). 

MKl  1 7  Communicator  --  It  handles  all  communications  with  the  MK117  fire 
control  system.  Its  function  is  to  read  data  from  the  MKl  17  to  translate 
that  data  from  the  MKl  17  UYK-7  format  to  VAX  data  format,  and  to  relay  the 
appropriate  data  packets  to  other  SMARTTS/VAX  software  components. 

MSCG  Communicator  —  it  handles  the  communications  between  the  VAX  and 
the  MSCG,  translating  the  data  for  use  by  the  MSCG  or  VAX  as  necessary. 

Bulletin  Board  —  It  is  a  data  area  that  contains  information  relevant 
to  the  status  of  the  entire  SMARTTS  system  (i.e.,  subsystem  operating 
states,  hardware  availabilities,  and  other  globally  known  information).  It 
stores  this  information  and  makes  it  accessible  to  all  SMARTTS/VAX  software 
functions. 

Command  Net  --  It  allows  communication  among  SMARTTS  software  components 
by  relaying  commands  from  software  components  acting  as  command  sources  to 
software  components  acting  as  command  sinks.  It  also  contains  the 
definitions  and  formats  for  all  messages  used  in  the  system,  thus  enforcing 
command  exchange  protocols  among  SMARTTS/VAX  software  components. 
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System  Services  —  It  contains  a  set  of  general  purpose  routines  used  by 
other  modules,  such  as  opening  and  closing  files,  coordination  of 
intercomponent  executions,  and  control  access  to  shared  and  private 
computational  facilities  and  resources. 
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APPENDIX  C 

SELECTED  SMARTTS  REPORTS 
DEVELOPED  BY  ECLECTECH  ASSOCIATES,  INC. 

(H)  Hardware  Maintainability  Plan  (March  1981)  -  This  document  outlines  the 
aspects  of  maintainability  program  management,  equipment  design, 
maintenance  philosophy,  sparing  and  documentation,  organizational 
resources,  maintenance  activity  monitoring,  and  general  status  reporting. 

(H)  Maintenance  Manual  (September  1982)  -  This  document  identifies  the 

hardware  elements  of  the  system.  It  is  also  a  guide  to  repair  the 
SMARTTS  system. 

(H)  Trainer  Engineering  Report  (February  25,  1981)  -  This  document  contains 
initial  design,  performance  and  operational  characteristics  of  the 
preprototype  version  of  SMARTTS.  It  also  defines  how  these 

characteristics  and  this  preprototype  version  of  SMARTTS  will  (a) 
interface  with  current  SCST  facilities  at  Norfolk,  and  (b)  affect  the 
SCST  facility,  the  instructors,  operators  and  trainees. 

(S)  Data  Base  Design  Document  (DBDD)  (November  1982)  -  This  document 

describes  the  SMARTTS  data  base  and  its  format. 

(S)  Integrated  Logistics  Support  Requirements  Report  (ILS  Requirements 
Report)  (February  198) )  -  this  document  identifies  requirements  and 
recommendations  that  will  lead  to  the  continuous  improvement  of  SMARTTS 
operational  capability  and  availability  over  SMARTTS'  life  cycle. 

(S)  Interface  Design  Specification  (IDS)  (October  1982)  -  This  document 

shows  data  that  is  passed  between  the  SMARTTS  VAX  and  SMARTTS/MSCG. 

(S)  Program  Description  Document  (PDD)  (November  1982)  -  This  document  gives 
a  listing  of  the  SMARTTS  document  headers  along  with  the  source  code. 

(S)  Program  Design  Specification  (PDS)  (November  1982)  -  This  document  gives 
the  internal  description  of  the  SMARTTS  software. 

(S)  Program  Performance  Specification  (PPS)  (July  1981)  -  This  document 

describes  the  design  of  the  SMARTTS  system. 

(T)  Instructor  Course  Instructor  Lesson  Guide  (November  1981)  -  This 

document  contains  lesson  guides  for  each  day  of  the  SMARTTS  instructor 
courses  along  with  handouts  for  the  course. 

(T)  Instructor  Handbook,  Device  Utilization  Manual,  Volumes  I  and  II 

(October  1981 ,  Volume  I  and  November  l98l ,  Volume  IT)  -  Volume  T 
contains  instructional  techniques.  Volume  II  contains  exercise  lessons. 

(T)  Maintenance  Course  Instructor  Lesson  Guide  (July  1982)  -  This  document 
contains  notes  and  course  outlines  Tor  teaching  the  maintenance  course. 
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(T)  Operator  Course  Instructor  Lesson  Guide  (May  1982)  -  This  document 

contains  notes  and  course  outlines  for  teaching  the  instructor  course. 


(T)  Operator's  Manual  (May  1982)  -  This  document  describes  how  to  operate 
the  SMARTTS  system. 


NOTE:  (H)  =  Hardware 

(S)  =  Software 

(T)  =  Training 
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ACRONYMS  FOR  SMARTTS 


AIT  Automatic  Interactive  Target 

CMI  Computer  Managed  Instruction 

CRC  Classroom  Remote  Console 

IRC  Instructor  Remote  Console 

ISO  Instructional  Systems  Development 

KAST  Kalman  Automatic  Sequential  Target  Motion  Analysis  Technique 

MSCG  Main  Simulation  Computer  Group 

NTEC  Naval  Training  Equipment  Center 

NWP  Naval  Weapons  Publication 

PCO  Prospective  Commanding  Officer 

PI  Performance  Indicator 

PXO  Prospective  Executive  Officer 

SAT  Systems  Approach  to  Training 

SCST  Submarine  Combat  System  Trainer 

SMARTTS  Submarine  Advanced  Reactive  Tactical  Training  System 

SNR  Signal  to  Noise  Ratio 

SOAC  Submarine  Officer  Advanced  Course 

S08C  Submarine  Officer  Basic  Course 

SOIC  Submarine  Officer  Indoctrination  Course 

TAT  Training  Assistance  Technology 

TEF  Tactical  Effectiveness  Factor 

TMA  Target  Motion  Analysis 

TSA  Training  System  Analysis 

TSET  Training  System  Evaluation  Test 

TSP  Time  to  Station 


NAVTRAEQUIPCEN  80-C-0079 
DISTRIBUTION 


Commanding  Officer 

Naval  Training  Equipment  Center  (N71) 
Orlando,  FL  32813  50 

Defense  Technical  Information  Center 
Cameron  Station 

Alexandria,  VA  22310  12 

(All  others  receive  one  copy) 

Director 

US  Army  Research  &  Technology  Lab 
DAVDL-AS  (ATTN:  R.  S.  Dunn) 

207  S.  Ames  Research  Center 
Moffett  Field,  CA  94035 

Mr.  Barker 

US  Army  Field  Artillery  School 
ATSF  DS 

Ft.  Sill,  OK  73503 
Commandant 

US  Army  Command  &  General  Staff 
College 
Library  Div. 

Ft.  Leavenworth,  KS  66027 

Commanding  Officer 
Redstone  Scientific  Info  Center 
US  Army  Missile  Command 
ATTN:  DRSMI-RPR  (Documents) 

Redstone  Arsenal ,  AL  35898 

ATSK-AAC/Leon  Jones 
Commandant 

US  Army  Missile  and  Munitions  Center 
&  School 

Redstone  Arsenal ,  AL  35897 

Conrad  Technical  Library 

US  Army  Signal  Center  &  Ft.  Gordon 

Bldg.  29807 

Ft.  Gordon,  GA  30905 

Army  Training  Support  Center 
ATIC-DST-P 

Ft.  Eustis,  VA  23604 

CDR.  US  Army  Mobility  Equipment 

R&D  Command 

ATTN:  Library 

Ft.  Bel  voir,  VA  22060 


Chief,  Research  &  Studies  Office 
Office  Deputy  Chief  of  Staff 
for  Personnel 
Dept,  of  Army 
Washington,  DC  20310 

Arthur  W.  Lindberg 

US  Army  Avionics  R&D  Activity 

DAVAA-C 

Ft.  Monmouth,  NJ  07703 

Commander,  USAISD 
ATTN:  ATSI-EDT-L 
Ft.  Devens,  MA  01433 

Dr.  Gerhard  Eichweber 
Kurt-Eichweber  Prazisionsgeratewerk 
Schuetzenstr  75-85 
2000  Hamburg  50 
W.  Germany 

COL  D.  J.  Walters 

National  Defense  Headquarters 

DOGC 

Ottawa,  Ontario  Canada  K1A  0K2 

Bulletin  Editor 
Human  Factors  Society 
P.  0.  Box  1369 
Santa  Monica,  CA  90406 

Gary  Poock 
Naval  PG  School 
Code  55PK 

Monterey,  CA  93940 
M.  McDowell 

Navy  Personnel  R&D  Center 
Library,  Code  P202L 
San  Diego,  CA  92152 

Commanding  Officer 
Navy  Personnel  R&D  Center 
ATTN:  Dr.  James  Tweeddale 
San  Diego,  CA  92152 

Commanding  Officer 

Fleet  Combat  Training  Cen,  Pacific 

Code  09A 

San  Diego,  CA  92147 


NAVTRAEQUIPCEN  80-C-0079 


Commanding  Officer 
Fleet  Anti-Submarine  Warfare 
Training  Center,  Pacific 
Code  003 

San  Diego,  CA  92147 
Commanding  Officer 

Fleet  Combat  Training  Center,  Pacific 
Code  04 

San  Diego,  CA  92147 

Commanding  Officer 
COMSUBDEVRON  Twelve 
US  Naval  Base,  New  London 
Groton,  CT  06340 

Commanding  Officer 

Naval  Underwater  Systems  Center 

New  London,  CT  06320 

Commanding  Officer 

Naval  Underwater  Systems  Center 

Newport,  RI  02840 

Commanding  Officer 

Fleet  ASW  Training  Center,  Atlantic 

Naval  Base 

Norfolk,  VA  23511 

Commander 

Submarine  Group  Two 

USN  Submarine  Base,  New  London 

Groton,  CT  06340 

Commanding  Officer 
US  Naval  Submarine  School 
Code  10 

USN  Submarine  Base,  New  London 
Groton,  CT  06340 

Commanding  Officer 

Fleet  Training  Center 

ATTN:  Training  Resources  Dept. 

Naval • Station 
San  Diego,  CA  92136 

Commanding  Officer 

Naval  Educ.  &  Train.  Support  Cen. 

Code  N-7  (Gloria  Houston) 

San  Diego,  CA  92132 

Commanding  Officer 

Naval  Educ.  &  Train.  Support  Cen. 

Pacific 

Code  N5B  (Mr.  McElroy) 

San  Diego,  CA  92132 


Chief,  Naval  Educ.  &  Training 
Liaison  Off. 

Human  Resource  Lab. 

Operations  Training  Div. 

Williams  AFB,  AZ  85224 

Chief  of  Naval  Air  Training 
Naval  Air  Station 
Code  N-301 

Corpus  Christi,  TX  78419 
CW02  Ray  Priest 

Naval  Air  Technical  Training  Cen. 

Code  7411 

NAS  Memphis  (85) 

Millington,  TN  38054 

Commanding  Officer 

Naval  Air  Technical  Training  Cen. 

Code  104,  Bldg.  S-54 

NAS  Memphis  (85) 

Millington,  TN  38054 

Chief  of  Naval  Education  &  Training 
Pensacola,  FL  32508 

Officer  in  Charge 

US  Naval  Submarine  Training  Center, 
Pacific 

San  Diego  Department 
140  Sylvester  Rd. 

San  Diego,  CA  92106 

Chief  of  Naval  Education  &  Training 

Code  0112 

NAS 

Pensacola,  FL  32508 

Chief  of  Naval  Education  &  Training 
Code  00A 

Pensacola,  FL  32508 

Chief  of  Naval  Education  &  Training 
Code  N- 21 

Pensacola,  FL  32508 

Chief  of  Naval  Education  &  Training 
Code  N-2 

Pensacola,  FL  32508 

LSO  Training  Model  Manager 
Officer  in  Charge 
LSO  School 
Box  171 

NAS  Cecil  Field,  FL  32215 


NAVTRAEQUIPCEN  80-C-0079 


Commander 

Naval  Air  Force 

US  Atlantic  Fleet  (Code  312E) 

NAS  Norfolk 
Norfolk,  V A  23511 

Commander 

Fleet  Combat  Training  Cen.,  Atlantic 
ATTN:  Mr.  Hartz,  Code  20A 
Dam  Neck 

Virginia  Beach,  VA  23461 
Commander 

Naval  Surface  Weapons  Center 
ATTN:  Library,  RM  #1-321 
Silver  Spring,  MD  20910 

Richard  Walchli 
Naval  Air  Test  Center 
SY70A 

Patuxent  River,  MD  20670 
Commander 

Fleet  Combat  Training  Cen.,  Atlantic 
ATTN:  Tactics  Training  Office 
Dam  Neck 

Virginia  Beach,  VA  23461 
Commander 

Fleet  Combat  Training  Cen.,  Atlantic 
ATTN:  TAO  Training  Office 
Dam  Neck 

Virginia  Beach,  VA  23461 
Commander 

Naval  Air  Test  Center 
CT252 

Patuxent  River,  MD  20670 
Commander 

Naval  Sea  Systems  Command 
Code  335 

Washington,  DC  20362 
Commander 

Naval  Sea  Systems  Command 
PMS-403 

Washington,  DC  20362 
Commander 

Naval  Sea  Systems  Command 
ATTN:  PM- 2 
Washington,  DC  20362 


Commander 

Naval  Sea  Systems  Command 
ATTN:  PMS-396 
Washington,  DC  20362 

Commander 

Naval  Sea  Systems  Command 
Technical  Library 
SEA  09B312 

Washington,  DC  20362 
Commander 

Naval  Air  Systems  Command 
Washington,  DC  20361 

Commander 

Naval  Air  Systems  Command 
Weapons  Training  Div. 

AIR  413 

Washington,  DC  20361 

George  Tsapaas 

Naval  Air  Systems  Command 

AIR  340D 

Washington,  DC  20361 

Chief  of  Naval  Operations 
0P-39T 

Washington,  DC  20350 

Chief  of  Naval  Operations 
OP-29 

Washington,  DC  20350 

Chief  of  Naval  Operations 
0P-593B 

Washington,  DC  20350 

Chief  of  Naval  Operations 
OP- 59 

Washington,  DC  20350 

Chief  of  Naval  Operations 
OP-596 

Washington,  DC  20350 

Dr.  R.  G.  Smith 

Chief  of  Naval  Operations 

0P-987H 

Washington,  DC  20350 

Chief  of  Naval  Operations,  OP-115 
Research,  Development  &  Studies 
Room  6836 

Washington,  DC  20350 


3  of  5 


*'  /*  i1*  •'»  .  •  »'•  #\ 

*  V  *  m  •  .  t  _  ■  *  ft  -  V  1 


.  .  .\V  \V.V 


NAVTRAEQUIPCEN  80-C-0079 


*d 


LT  Thomas  N.  Crosby 
Naval  Air  Systems  Command 
Code  531 3X 

Washington,  DC  20361 
Commander 

Naval  Air  Development  Center 
ATTN:  Dr.  Edgar  Johnson 
Technical  Director 
Warminster,  PA  18974 

Air  Force  Flight  Test  Center 
Technical  Library/Stop  238 
Edwards  AFB,  CA  93523 

Dr.  Marty  Rockway 
AFHRL/OT  (UDRI ) 

Williams  AFB,  AZ  85224 

US  Air  Force  Human  Resources  Lab. 
AFHRL-OT 

Operations  Training  Div. 

Williams  AFB,  AZ  85224 

Chief 

Human  Resources  Lab. 

Operations  Training  Div. 

Williams  AFB,  AZ  85224 

Superintendent 

US  Air  Force  Academy  Lib. 

Code  DFSEL-D 

Colorado  Springs,  CO  80840 

Dr.  Joseph  Yasutake 
AFHRL/ID 

Lowry  AFB,  CO  80230 

COL  Robert  D.  O'Donnell 
AFAMRL/HEG 

Wright-Patterson  AFB,  OH  45433 

Human  Factors  Branch 
AFAMRL 

Wright-Patterson  AFB,  OH  45433 
Director 

Air  University  Library 
ATTN:  ALU/LSE 
Maxwell  AFB,  AL  36100 

HQAFSC/DLS 

Andrews  AFB,  DC  20334 

MAO  Bob  Croach 
HQ  USAF/XOOTD 
Washington,  DC  20330 


4  of  5 


Rome  Air  Development  Center 
ATTN:  Documents  Section  (TSTD) 
Griffiss  AFB,  NY  13441 

David  C.  Nagel 

NASA  Ames  Research  Center 

LM-239-3 

Moffett  Field,  CA  94035 

Edward  M.  Huff,  Chief 
Helicopter  Human  Factors  Office 
Mail  Stop  239-21 
NASA  Ames  Research  Center 
Moffett  Field,  CA  94035 

NASA  Langley  Research  Center 
Technical  Library,  MS  185 
ATTN:  Document  Cataloging 
Hampton,  V A  23665 

Technical  Library 

Chemical  R&D  Center 

Aberdeen  Proving  Ground,  MD  21010 

USA  Ordnance  Cen.  &  School  Library 
Bldg.  3071 

Aberdeen  Proving  Ground,  MD  21005 

Dept,  of  Transportation  Library 
FOB-1 OA  Services  Branch,  M-493.2 
800  Independence  Ave.,  SW 
Washington,  DC  20591 

Robert  T.  Hennessy,  COM  ON  HUM  FAC 
National  Research  Council 
2101  Constitution  Ave.,  NW 
Washington,  DC  20418 

M.  Stephen  Huntley 
Dept,  of  Transportation 
DQT/TSC-45 
Kendall  Square 
Cambridge,  MA  02142 

Jackson  W.  Royal 
Dept,  of  Transportation 
DOT/TSC-54 
Kendall  Square 
Cambridge,  MA  02142 

Commandant  of  Marine  Corps 
Code  TDA-50 
Washington,  DC  20380 

Commanding  General 

US  Marine  Corps  Dev.  &  Educ.  Cmd. 

ATTN:  Aviation  Div. 

Quantico,  VA  22134 


NAVTRAEQU I PCEN  80-C-0079 


Commanding  Officer 
NAVSUBTRACEN,  PAC 
ATTN:  Carl  Thomas 
Pearl  Harbor,  HE  96860 

Commanding  Officer 
FL  E  BALM  I S  liBTRAC  EN 
ATTN:  CDR  Peter  Thomas 
Charleston,  SC  29408 

Commanding  Officer 
TRITRAFAC,  Bangor 
ATTN:  Code  40 
Bremerton,  WA  98315 

Commanding  Officer 
COMSUBLANT 
ATTN:  N241 
Norfolk,  VA  23511 

Commanding  Officer 
COMSUBPAC 

Pearl  Harbor,  HI  96860 

Strategic  Systems  Project  Office 
Training  Branch 
ATTN:  George  Ball 
Washington,  DC  20376 

COL  Gerald  A.  Blake 
Deputy  for  Simulators 
Simulation  System  Project  Office 
Wright-Patterson  AFB,  OH  45433 


5  of  5 


